102 - Babelway - Basic Transformation

The 102 - Basic Transformation Babelway course will cover the basics of the transformation tool in Babelway. This will equip you with the knowledge to build your own channels and transformations.

Agenda includes:
Transformation
Drag and Drop
Easy Function Editor
Test Cases
Loops
Task #1: Create a channel, map all fields
Task #2: Use functions date change and concat, make a test case

Table of contents
Transformations
Understanding Transformation (1:07)
Message Representation
Drag and Drop Mapping (1:04)
Loops
Understanding Loops
Mapping Loops (4:11)
Adding Loops
Task #1
Functions
Understanding Functions
Functions in Practice (4:44)
Test Cases
Using Test Cases
Task #2

Transformations

Understanding Transformation (1:07)

Message Representation

Tree Format Babelway represents each messages type in a 'tree' format. This gives you a visual representation of each field of data within a file.

For example, an XML file may look like this:

If you were to upload it into Babelway, it would be represented like this:

Each structure is a little different:

X12:

CSV:

In a later lecture, we'll go over visual transformation and you'll start to see how these tree structures interact with each other. The idea behind message definitions is that each node represents some data within the file. Isolating each piece of data gives you the ability to map it to other tree structures and manipulate that data with functions.

Data Nodes

Within the tree structure, Each line/square represents some data. The color of the square identifies what kind of data it is:

Blue Square - Data Node, changes with each message

Green Square - Data Node, default or expected value

White Square - Group Node, structural

Typically, you will map blue and green square nodes. An example of a blue node might be an Invoice Number, something you expect to change with each new file. An example of a green node might be a Document Type Code, something you might expect to be the same from one file to the next. An example of a white node might be the Invoice group, a structural data node that encompasses all the data for one invoice.

Drag and Drop Mapping (1:04)

Loops

Understanding Loops

Loops are an essential part of Babelway mappings. Loops are used when White square group nodes repeat within a file. Examples of when loops are used:

Multiple invoices in one file
Multiple line items on an invoice
Multiple sub-totals in a line item

Whenever you use data within the structural group, you must map the loop above it.

Here is an XML Invoice file with repeating groups:

You can see that the tag <invoice> and <lineItem> repeats within the file. Thus, loops must be created on these group nodes. The file looks like this in Babelway:

You can see that there are 'Loop' nodes created above the <invoice> group node and the <lineItem> group node.

These loops will be mapped to loops on the outgoing message:

This document is attached to the course.

Multiple XML invoice.xml

<request>
<invoice>
	<invoiceHeader>
		<invoiceNumber>INV123</invoiceNumber>
		<invoiceDate>20160301</invoiceDate>
		<poNumber>4500064817</poNumber>
	</invoiceHeader>
	<invoiceDetail>
		<lineItem>
			<quantity>1</quantity>
			<unit>EA</unit>
			<unitPrice>1.99</unitPrice>
			<description>Nails</description>
		</lineItem>
	</invoiceDetail>
</invoice>
<invoice>
	<invoiceHeader>
		<invoiceNumber>INV124</invoiceNumber>
		<invoiceDate>20160301</invoiceDate>
		<poNumber>4500064817</poNumber>
	</invoiceHeader>
	<invoiceDetail>
		<lineItem>
			<quantity>10</quantity>
			<unit>EA</unit>
			<unitPrice>21.99</unitPrice>
			<description>Tape</description>
		</lineItem>
		<lineItem>
			<quantity>1</quantity>
			<unit>EA</unit>
			<unitPrice>104.99</unitPrice>
			<description>Drill</description>
		</lineItem>
				<lineItem>
			<quantity>7</quantity>
			<unit>EA</unit>
			<unitPrice>20</unitPrice>
			<description>Boards</description>
		</lineItem>
	</invoiceDetail>
</invoice>
</request>

Mapping Loops (4:11)

Adding Loops

Sometimes, it may be necessary to add loops manually. Examples of this are XML files uploaded via a sample file, or some flat file definitions.

This can be done in the Message In or Out tab. In this example XML, we see that a tree has been created for all parts of the file:

For the purpose of mapping, we don't need multiple tree groups for each invoice and each lineItem, so we can delete them to get a clean sample. We choose the node that we want to delete, and press the 'delete' on the bottom right:

After that, we can manually create loops on the group nodes that repeat (in this case invoice and lineItem):

Task #1

Task #1.1

Multiple XML invoice.xml

<request>
<invoice>
	<invoiceHeader>
		<invoiceNumber>INV123</invoiceNumber>
		<invoiceDate>20160301</invoiceDate>
		<poNumber>4500064817</poNumber>
	</invoiceHeader>
	<invoiceDetail>
		<lineItem>
			<quantity>1</quantity>
			<unit>EA</unit>
			<unitPrice>1.99</unitPrice>
			<description>Nails</description>
		</lineItem>
	</invoiceDetail>
</invoice>
<invoice>
	<invoiceHeader>
		<invoiceNumber>INV124</invoiceNumber>
		<invoiceDate>20160301</invoiceDate>
		<poNumber>4500064817</poNumber>
	</invoiceHeader>
	<invoiceDetail>
		<lineItem>
			<quantity>10</quantity>
			<unit>EA</unit>
			<unitPrice>21.99</unitPrice>
			<description>Tape</description>
		</lineItem>
		<lineItem>
			<quantity>1</quantity>
			<unit>EA</unit>
			<unitPrice>104.99</unitPrice>
			<description>Drill</description>
		</lineItem>
				<lineItem>
			<quantity>7</quantity>
			<unit>EA</unit>
			<unitPrice>20</unitPrice>
			<description>Boards</description>
		</lineItem>
	</invoiceDetail>
</invoice>
</request>

Basic CSV Invoice.csv

Invoice Number,Invoice Date,PO Number,Quantity,Unit,Unit Price,Item Description
TEST01,11/23/2015,4500237392,10,,299.99,Ski Boots
TEST01,11/23/2015,4500237392,1,,19.99,Marketing Materials
TEST02,12/1/2015,4508390298,10,,49.99,Ski Poles

Create a new channel, Email Gateway In, FTP Server Gateway Out, XML Message In, CSV Message Out
Put the BasicTransformationTask11 name.

Email Address: basictransformation11@eu1.babelway.net

User: juan.pablo.perez Password: password Directory: /basictransformation11

Select Multiple XML invoice.xml

Select Basic CSV Invoice.csv

Create Loops on XML Message

It already has the loops

Create Visual Transformation

Map all fields

Map XML Loops to CSV Loop

Save and test if it works.

Send the email

Check if the process has been executed on Babelway.

Change the email address to put properly basictransformation11@eu1.babelway.net

Send the email again.

Check if the process has been executed with success:

Open the message out:

Multiple XML invoice.csv

Invoice Number,Invoice Date,PO Number,Quantity,Unit,Unit Price,Item Description
INV123,20160301,4500064817,1,EA,1.99,Nails
INV124,20160301,4500064817,10,EA,21.99,Tape

It is not correct because we need another loop for each detail

Send the email again:

Check if it has been processed correctly

It is not working again.

The detail fields were removed. We have to add them again:

Send the email again

Check if it has been processed correctly this time

It is not working again!

Task #1.2

Simple XML invoice.xml

<request>
<invoice>
	<invoiceHeader>
		<invoiceNumber>INV123</invoiceNumber>
		<invoiceDate>2016-03-01</invoiceDate>
		<poNumber>4500064817</poNumber>
	</invoiceHeader>
	<invoiceDetail>
		<lineItem>
			<quantity>1</quantity>
			<unit>EA</unit>
			<unitPrice>1.99</unitPrice>
			<description>Nails</description>
		</lineItem>
	</invoiceDetail>
</invoice>
</request>

Basic X12 Invoice.txt

ISA*00*          *00*          *01*ABCCOM         *12*9999999999     *101205*1710*U*00401*000001320*0*P*>
GS*IN*4405197800*999999999*20101205*1710*1320*X*004010VICS
ST*810*1004
BIG*20101204*217224*20101204*P792940
REF*DP*099
REF*IA*99999
N1*ST**92*123
ITD*01*3***0**60
IT1*1*4*EA*8.60**UP*999999330023
PID*F****Item Description
SAC*C*D240***100
TDS*21740
CAD*****GTCT**BM*99999
CTT*8
SE*13*1004
GE*1*1320
IEA*1*000001320

Create another new channel, X12 Message In, XML Message Out
Put the BasicTransformationTask11 name.

Select Basic X12 Invoice.txt

Select Simple XML invoice.xml

Create Loops on XML Message

Create Visual Transformation

Map all fields on XML (some investigation required!)

Map X12 Loops to XML Loops

If you need access to the Babelway platform, you can signup for a free trial by going to www.babelway.net and registering:

Functions

Understanding Functions

Sometimes when mapping data, it is not a direct 1 to 1 mapping per field. In these cases, you can apply functions to manipulate data. Examples where functions are needed are:

Change to data formats (like with a date field)
Using multiple data nodes for one map
Using arithmetic
Conditional mapping behavior
Using part of a data node

Functions in Babelway

Babelway functions are based on Xpath. Some functions are standard Xpath functions, while others are specific to Babelway. It is possible to use multiple functions per field if needed. You can use the 'Function Bar' or the 'Easy Function Editor'

When using functions, you define the function you want to use and then input data for each parameter. Nodes from the tree can be dragged to the parameter:

The most commonly used functions in Babelway are:

changeDateTimeFormat()
concat() - for combining multiple fields
Arithmetic: + - * div
IF() - conditional check on data
Substring() - Using part of a field's data
currentDateTime() - supply the current time in format yyyy-MM-dd HH:mm:ss.SSS

Functions in Practice (4:44)

Test Cases

Using Test Cases

Test Cases are used to verify your mapping setup. They run through the latest 'Save' (not the last deployment). This gives you the ability to test your mappings without deploying a channel into a live state.

Test cases also show up in the monitoring section, so you can verify their successful processing status. They provide an easier way to testing channels than using external gateways (like having to upload a file to an FTP site just to test the mapping).

To create a Test case, click on 'Test cases' on the upper right hand side of the page:

Then click on 'Add a test case'

Then, upload a sample file you wish to test:

Once uploaded, you can click 'Run Test Case' to start the test. The result will show you success or failure and give you access to download the output file to verify the results.