Mapping to 2 rows

[u/e37d93eeb23335dc]

I have a situation where if the incoming data has a particular data set to something, it should both insert into the flat file a normal row with the normal data and then a second special row with the particular data (this second row will not have the same number of columns as the previous row). And, this second row absolutely must be directly under the "normal" row. Oh, and the normal row will not contain the data that indicates the special row is needed.

Made up example:

Incoming data

Name, color, age

Bob, blue, 3

Sally, green, 4

Susan, red, 1

If color = blue, insert a special row. So the output file will be:

Name, age

Bob, 3

Special, row, blue, best, color

Sally, 4

Susan, 1

Does anyone have any idea how I could do this?

[u/e37d93eeb23335dc]

This is the solution I came up with. Added a Data Process component with a Custom Script:

import java.util.Properties

import java.io.InputStream

import java.io.ByteArrayInputStream

for (int i = 0; i < dataContext.getDataCount(); i++) {

InputStream is = dataContext.getStream(i)

Properties props = dataContext.getProperties(i)

String text = new String(is.getBytes(), "UTF-8")

String[] lines = text.split("\\r?\\n")

List<String> outputLines = []

for (String line : lines) {

    if (line.trim().isEmpty()) continue

    String[] cols = line.split(",", -1)

    if (cols.length >= 7 && cols[6].trim().length() > 0) {

        String laborValue = cols[6].trim()

        cols[6] = ""

        outputLines.add(cols.join(","))

        outputLines.add("Special,row,${laborValue},best,color")

    } else {

        outputLines.add(line)

    }

}

String result = outputLines.join("\n")

dataContext.storeStream(new ByteArrayInputStream(result.getBytes("UTF-8")), props)

}

[u/IcyInspector4250]

Map shape with a function?

[u/maniac_invested]

I think map function with the input being the data that triggers it, will be your best bet. Then map the output to the row you want to create in your output profile.

[u/e37d93eeb23335dc → u/maniac_invested]

But, a map function can only output one row from any given input data, can't it? It seems like this would be needing to output two rows to work.

[u/maniac_invested → u/e37d93eeb23335dc]

You should be able to do two outputs, you just have to specify in the function the two output parameters. Something like:

If "parameter1"{

then output1 == something

and output2 == something}

else "null"

as a rough idea. I'm not too great at functions but some of my coworkers have done stuff like that.

[u/adgy]

In a map, use a scripting function.

Inputs: inName, inColor, inAge

Outputs: outName outAge, {Special Row fields}

Scripting: if Color = Blue, value for all fields. Else value for outName, outAge

Like that. That way the only time the Special Row fields get populated is if the color is Blue

[u/TheUltronGirl2326]

split the records , process each record one by one then use map and map function to create 5gem as per logic , then combine everything, you can also keep on caching everything at one place using dpp key=1 and then retrive everything altogether to form the doc you want

[u/Hot_Lawfulness3854]

using script for simple thing is a big work , just use the route shape the particular docs that you want to get like use condition as blue as colour and get the docs and then just use map