A variable’s purpose or intention is a helpful suggestion within a variable’s name. These can include metrics, various transformation, function, and operation cases, and configuration.
Metrics are key performance indicators. Two common desirable measures are the processing duration (timer) and the number of records processed. Since these are metrics I always like to collect, I use scripts to manage specific variables to read and write these values as needed.
The Set() and Get() functions allow me to declare global variable “names” at run-time. For example, a timer Script might be
// Timer is a utility script to start, reset or collect a "timed" series of one or more Operations // or, perhaps Scripts, with Operations. _timer_name = IfEmpty(_1,"DEFAULT_TIMER_NAME"); _reset = Bool(IfEmpty(_2, false)); //optional, resets timer _seconds = Bool(IfEmpty(_3, false)); // optional, return seconds time = Get(_timer_name); If(IsNull(time) || Length(time)==0 || _reset, //initialize timer Set(_timer_name,Now()) //return Now() , If(_seconds, Now()-time, //return duration in seconds FormatDate(Now()-time,"HH:MM:SS") //or HH:MM:SS ); );
Typically, it’s used to record data pipeline execution times and compare variances in each execution time.
By initializing the timer in an Operation’s Setup Script (best), one can collect the duration within the Operation’s Wrapup Script.
The only required parameter is the name of the global variable (a timer name) used to keep multiple timers unique.
This global variable is holds the timer’s “start time”.
Future executions of Timer will return the time since initilized in (by default) HH:MM:SS format
Two other parameters are optional:
* Argument 2 (reset): boolean to “reset” the timer (reset global variable to “current time”).
* Argument 3 (seconds): boolean to return the duration since initialization in “seconds”.
Use “” for reset value (arg 2) if not resetting the timer but need “seconds” since initialization
// initialize; returns 00:00:00 but this first return value is not useful RunScript("<TAG>Scripts/Utilities/Timer</TAG>",":mytimer"); sleep(9); duration = RunScript("<TAG>Scripts/Utilities/Timer</TAG>",":mytimer"); // Returns 00:00:09
Note, the chosen timer name is a global variable holding the initialization time so $:mytimer holds the initialization time.
Note the colon (:) in the duration variable naming convention. The 2nd and 3rd parameters are optional and allow me to reset the timer or return the duration in seconds in later calls. Use of the _seconds boolean value is useful if you need to give an average time per record. For that, you also need a count value.
A simple counter naming convention might be $#counter. Increasing its values is simple as
We’ll offer a Script in a future post for a robust, thread-safe counter.
Various reporting cases use Status values or “whether to proceed.” I like to use $?status for my status variables, and it has a boolean or some small integer value with a specific meaning. Sometimes a “simple” status is not enough, and you might want to augment the status with more information (like a message). In cases like this, I use a $[0-9]MsgName naming convention. The use of an integer value at the front of the global variable name tells me it is a message string (and a message severity). I also vary the integer in the name if I “bubble-up” messages to log child messages into a “parent Operation” log.
Transformations, Functions, and Operations
There are three use cases where you might, or have to use one or more Global variables.
Value re-use in a Transformation
Within one transformation, one might map one source field to two or more target fields. The transformation logic may be unique for each. Here, you have a choice:
1. assign the first map to a global and use the global for the rest or
2. map the same source in each transformation
Your first inclination should be the latter choice. This way, the mapping is visible in the Transformation image. (Remember to use CacheLookup() or SfCacheLookup() to save on time-expensive database queries within one transformation). The first choice may be justifiable if you need to reuse a ‘calculated’ value. Here’s where a transformation-only global naming convention is a good idea. I use the @-sign (as in $@transVarName) to distinguish transformation-only globals.
Function Requires a Global Variable
Function RegExMatch() and most use cases of SetInstances()/GetInstance() must use a Global. If used only within a Transformation, the earlier suggestion of $@transVarName works well. If I find many required global cases outside of Transformations, I’ll use $.funcOname for a global naming style. Note the use of the leading period.
I/O for Utility Operations
Utility Operations called multiple times are not uncommon. Parameterized input make these operations more robust. Operations only return a boolean indicating success or failure. If you need non-boolean result types as the outcome, a Global variable is necessary.
I use $_OpIname as input parameters (a leading underscore like Script parameters) for these utility Operations. For the non-boolean result, I use $%OpOname for output, including filenames. If some messaging is also needed, I use $`OpOname for a global variable naming style.
If there’s any naming confusion, choose to focus on where the variable is read when choosing any name.
A configurable data pipeline makes for a more robust pipeline. Variables are an excellent place to record configuration details. Configuration is the motivation of the Jitterbit Project Variable.
A Jitterbit Project Variable is a dedicated Jitterbit component. They are an excellent place for configuring your projects’ data pipelines. (Project variables are Global variables and appear green but bolded within Scripts.)
Also, Jitterbit’s Web Management Console (WMC) allows viewing and editing of Project Variables. WMC is beneficial for changing values of these “configuration” variables as one does not need to use Jitterbit Studio.
Component folders in Studio’s Design Tree help with the organization of your Project variables (or any component). That said, its best to use a hierarchical naming convention when naming your Project variables. Using a dot convention can work well here for your configuration variable list organization.
For example, with the Email Message component, one could use Project variables like the following. I like to use uppercase within Project variable names (out of habit) as I sometimes use the “dot” convention in other global variable names too.
- MAIL.notify.to, MAIL.notify.cc, MAIL.report.to
- MAIL.auth.account, MAIL.auth.password
It’s important to note that Project variable values are not available in the four test modes of Jitterbit Studio:
1. Test Connection
2. Test the Transformation
3. Test an Operation that Uses the Transformation
4. Test the Web Service Call
Here, one should use defaults in
// Scripts $ProjectVar = IfEmpty($ProjectVar, "defaultVal");
or in Component definition fields
Migration from one environment to another can likely need some “re-configuration.” For example, your test FTP or File Share path or server might be different than your production endpoints.
When you “migrate” to your Production environment, you want to cut, if not drop, manual change for risk of breaking something. Agent Variables are useful here.
Each Jitterbit Agent has a configuration file called “jitterbit.conf.”
Within this configuration file, you can define Global variables available to all Projects running on that Agent. Within jitterbit.conf, see the section titled “[PredefinedServerGlobalDataElement].” The Agent then needs a restart, but you can define Global variables here for use with all your Projects running on this Agent.
Be careful with your use of the backslash character () as this character is also an escape character when reading from a file. For example, don’t use the backslash as the last character of a Windows path value as this will escape the trailing single-quote (‘).
For a naming convention, I like to lead with a tilde (~) character and use hyphens within the variable name for two reasons. The leading tilde immediately declares the variable as an Agent variable. Also, because of alphanumeric sorting in the Data Elements tab, the Agent Variables will appear above the Project Variables.
The hyphen use makes for a required, less-orthodox method to display Agent variable values. Endpoint credentials can be Agent variable values, but not recommended for Production environments.