Skip to content

oswaldobapvicjr/jep-data-extension

Repository files navigation

jep-data-extension

GitHub Workflow Status Coverage Maven Central javadoc

jep-data-extension is a project that extends JEP, the powerful mathematical expression parser and evaluator for Java, introducing data manipulation capabilities with custom functions and operators ready to use, including RegEx, XPath, JSONPath and Web Services.

For details about JEP core features, access JEP documentation.


New functions

Package Name Description
String functions camel Converts a string to camel case
concat Concatenates the arguments into a string.
endsWith Returns 1 (true) if a string ends with the specified suffix.
findMatch Returns the first match of the given regular expression found inside a string.
findMatches Returns a list containing all matches of the given regular expression inside a string.
formatString Returns a formatted string according to given pattern and variable arguments.
lpad/leftPad Left-pads a string to a given size.
lcase/lower Converts a text string to all lower-case letters.
matches Returns a 1 if the given string contains at least one match of a given RegEx.
normalizeString Normalizes a Unicode string, replacing accents and other diacritics with ASCII chars.
proper Converts a string to proper case, i.e., only the first letter in each word in upper-case.
replace Replaces all occurrences of a given string with another one.
replaceRegex Replaces all matches of a given regular expression.
rpad/rightPad Right-pads a string to a given size.
split Splits a string into a string array based on a separating string or regular expression.
startsWith Returns 1 if a string starts a given prefix.
trim Removes leading and trailing spaces.
ucase/upper Converts a text string to all upper-case letters.
Date functions now/sysdate Returns the system's current date & time.
str2date Converts a string into date by trying different patterns such as RFC-3339, RFC-822 and common ISO-8601 variations.
date2str Converts a date into string using a specified format.
daysBetween Returns the number of days between two dates.
endOfMonth Returns a date corresponding to the last day of the month given a source date.
isLeapYear Returns 1 if the given date or year number is a leap year, i.e., an year with 366 days.
year Returns the year for a given date.
quarter Returns the quarter of the year for a given date.
month Returns the month for a given date, a number starting from 1 (Jan) to 12 (Dec).
isoWeekNumber Returns the ISO week number for a given date.
weekday Returns the day of the week of a date, a number from 1 (Sunday) to 7 (Saturday).
day Returns the day of the month for a given date..
hour Returns the hour of day for a given date, as a number from 0 (12AM) to 23 (11PM).
minute Returns the minute within the hour for a given date, as a number from 0 to 59.
second Returns the seconds within the minute for a given date, a number from 0 to 59.
addDays Returns the result from adding a number of days to a date.
Math functions arabic Converts a Roman numeral to Arabic.
roman Converts an Arabic numeral to Roman.
Data manipulation & filtering xpath Returns the result of the given XPath expression at the specified XML.
jsonpath Returns the result of the given JSONPath expression at the specified JSON.
Statistical functions average Returns the average of the elements inside a given array, JSON array or collection (including valid representations).
count Returns the number of elements inside a given array, JSON array or collection (including valid string representations).
max Returns the largest value in a given array, JSON array or collection (including valid string representations).
min Returns the smallest value in a given array, JSON array or collection (including valid string representations).
Random uuid Produces a randomly generated type 4 UUID (Universally-unique Identifier) string.
Utility functions distinct Returns a list consisting of the distinct elements of a given list, array or JSON array.
getEnv Returns the value of an environment variable.
getSystemProperty Returns the value of system property.
isDecimal Returns 1 if the given parameter is a number (including strings) containing decimals.
isEmpty Returns 1 if a given parameter is either null or an empty String, JSON or collection.
isInteger Returns 1 if the given number (or valid string) represents a whole number, i.e, which lacks decimals.
readFile Returns the content of a text file in the file system.
typeOf/class Returns the Java type associated with the given variable.
Cryptography functions sha256 Computes the SHA-256 hash of the given string and transforms the binary result into a hexadecimal string.
toBase64 Encodes a string to Base64 encoding scheme.
fromBase64 Decodes a Base64 encoded string.
Web Services httpGet Returns data from a Web Service or RESTful API, as string.
http Invokes a specific method towards a Web Service/REST API.
httpHeader This function groups a variable number of HTTP header entries for usage with other HTTP functions.
basicAuthorizationHeader Generates a basic authorization header from a given username and password.
httpStatusCode Returns the HTTP status code of a given WebServiceResponse.
httpResponse Returns the HTTP response body from given WebServiceResponse, as string.

ℹ️ Find examples in the wiki.


Custom operators

JEP offers a good set of operators for common arithmetic and logical operations, for example: addition (+), subtraction (-), multiplication (*), division (/), modulus (%), power (^), "Boolean And" (&&), "Boolean Or" (||), and "Boolean Not" (!). These operators can be used when required and are supported by JEP.

Comparative operators

Standard JEP relational operators accept only numerical variables (including string representation of non-complex, numerical values). These operators have been overloaded to allow comparison of dates, as well as string representations of valid dates:

  • Less than (<)
  • Less than or equal to (<=)
  • Greater than (>)
  • Greater than or equal to (<=)
  • Equal (==)
  • Not equal (!=)

This allows the usage of dates in expressions that can be evaluated to true or false. For example:

if("2017-03-11T10:15:00:123Z" < now(), "past", "not past")

Element operator

Standard "element" operator ([]) was overloaded to support data extraction out of JSON arrays and other collections by index.

jsonArray[1] //extracts the first element of the given jsonArray

Note: Also achievable using the function get(jsonArray, 1).


How to include it

If you are using Maven, add jep-data-extension as a dependency to your pom.xml file.

<dependency>
    <groupId>net.obvj</groupId>
    <artifactId>jep-data-extension</artifactId>
    <version>1.0.7</version>
</dependency>

If you use other dependency managers (such as Gradle, Grape, Ivy, etc.) click here.


How to use it

Example 1: replacing characters from a text variable using the JEPContextFactory

  1. Add jep-data-extensions to your class path

  2. Put your source variables in a map:

    Map<String, Object> myVariables = new HashMap<>();
    myVariables.put("myText", "foo");
  3. Use the JEPContextFactory class to create an extended JEP object with all available extensions:

    JEP jep = JEPContextFactory.newContext(myVariables);

    Note: Alternatively, you may use the JEPContextFactory.newContext() to receive a new context with no initial variables. You may add them later, calling jep.addVariable(String, Object).

  4. Parse your expression:

    Node node = jep.parseExpresion("replace(myText, \"oo\", \"ee\")");
  5. Evaluate the expression with JEP's evaluate method:

    String result = (String) jep.evaluate(node); //result = "fee"
  6. Alternatively, you can use the assignment operator inside an expression. The new variable will be available in JEP context for reuse and can be retrieved using the getVarValue method:

    Node node = jep.parseExpression("myText = replace(\"fear\", \"f\", \"d\")"));
    jep.evaluate(node);
    String result = (String) jep.getVarValue("myText"); //result = "dear"

Example 2: comparing dates using the ExpressionEvaluatorFacade

The ExpressionEvaluatorFacade is a convenient object to quickly parse a single expression. A single call to the evaluate method will create a new evaluation context and return the expression result directly. It accepts a map with initial variables for use in the operation.

  1. Put your source variables in a map:

    Map<String, Object> myVariables = new HashMap<>();
    myVariables.put("date1", "2018-12-20T09:10:00.123456789Z");
  2. Evaluate your expression:

    String expression = "if(date1 < now(), \"overdue\", \"not overdue\")";
    String result = (String) ExpressionEvaluatorFacade.evaluate(expression, myVariables); 
    //result = "overdue"

Example 3: Consuming data from a RESTful API using the Evaluation Console

  1. Run jep-data-extension JAR file from a terminal:

    $ java -jar jep-data-extension-1.0.2.jar
    
  2. Consume the RESTful API with the httpGet function:

    JEP > dateTime = httpGet("http://date.jsontest.com/")
    {
       "time": "08:31:19 PM",
       "date": "05-12-2012"
    }
    
  3. Filter the JSON using a JsonPath expression that extracts the date field:

    JEP > date = jsonpath(dateTime, "$.date")
    05-12-2012
    
  4. Check the type of the generated date variable:

    JEP > class(date)
    java.lang.String
    
  5. Convert the string to Date:

    JEP > date = str2date(date, "dd-MM-yyyy")
    Wed Dec 05 00:00:00 BRST 2012
    
  6. Extract the week number from the converted date:

    JEP > isoWeekNumber(date)
    49