Skip to content
This repository was archived by the owner on Jan 21, 2024. It is now read-only.

Use custom deserializer and apply masking in place #29

Open
cut-to-the-cheese wants to merge 12 commits into
base: mule-4.x
Choose a base branch
from
Open

Use custom deserializer and apply masking in place #29

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
7a2b8a6
Use custom deserliazer and apply masking in place
May 11, 2021
fe52bab
Use custom deserliazer and apply masking in place
May 11, 2021
cbbbcc5
cleanup working point
Jun 3, 2021
ac03b7f
checkpoint
Jun 15, 2021
a40de01
checkpoint
Jun 15, 2021
ea126f8
checkpoint
Jun 16, 2021
68e2422
checkpoint
Jun 16, 2021
d65e90e
checkpoint
Jun 17, 2021
753f89d
checkpoint
Jun 17, 2021
014dc09
working point
Jun 18, 2021
880b9a3
Whole lotta more cleanup
Jun 29, 2021
b90a38e
Add unit tests
Jun 29, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 38 additions & 31 deletions json-logger/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,31 +1,38 @@
# Json-logger Extension

## 2.0.1 version - Release notes

Bug fixes:
* Added support for large payloads

## 2.0.0 version - Release notes

New features:
* External Destinations
* Data masking

Improvements:
* Field ordering

More details in the coming blog post (stay tuned!)

## 1.1.0 version - Release notes

New features:
* Scoped loggers to capture "scope bound elapsed time". Great for performance tracking of specific components (e.g. outbound calls)
* Added "Parse content fields in json output" flag so that content fields can become part of final JSON output rather than a "stringified version" of the content

Improvements:
* Removed Guava and caching in general with a more efficient handling of timers (for elapsed time)
* Optimized generation of JSON output
* Code optimizations
* Minimized dependency footprint (down from ~23MB to ~13MB)
* Optimized parsing of TypedValue content fields

# Json-logger Extension for Mule 4 runtime

Json logger makes uses of [Mule Java SDK](https://docs.mulesoft.com/mule-sdk/1.1/getting-started)
and contains a custom deserializer for masking fields in the JSON.

> Data masking is not possible in any other formats at this time e.g. xml, csv, etc.

## Serialization/Deserialization performance penalty
When the payload is passed into the json logger component as `application/json`, it is first deserialized
and masking if specified is applied in place. The serialization takes place when the log is being written out.
Therefore, at the very minimum, it is a two-step process. The transformation done before or after the json logger
to convert payload to `application/java` from `application/json` and vice versa will degrade performance.

## Operations:
There are two operations available:
1. logger: Use this for regular logging
2. loggerScope: Use this to measure response times for external calls

## Logging structure:
```json
{
"correlationId" : "cbfb9c1f-f904-40f2-bad9-2eac6bc05e84",
"message" : "",
"content": {
"dateOfBirth": "**-**-****"
},
"priority" : "INFO",
"tracePoint" : "START",
"locationInfo" : {
"fileName" : "api-main-router.xml",
"rootContainer" : "api-main",
"lineNumber" : "25"
},
"applicationName" : "proc-checkin-api-v1",
"applicationVersion" : "1.2.7",
"environment" : "local"
}
```
189 changes: 56 additions & 133 deletions json-logger/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,141 +3,64 @@
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

<modelVersion>4.0.0</modelVersion>
<groupId>ORG_ID_TOKEN</groupId>
<artifactId>json-logger</artifactId>
<version>2.0.1</version>
<packaging>mule-extension</packaging>
<name>JSON Logger - Mule 4</name>
<modelVersion>4.0.0</modelVersion>
<groupId>7abbfaeb-3816-4907-865e-76d7cb24120f</groupId>
<artifactId>json-logger</artifactId>
<version>3.0.0</version>
<packaging>mule-extension</packaging>
<name>JSON Logger - Mule 4</name>

<parent>
<groupId>org.mule.extensions</groupId>
<artifactId>mule-modules-parent</artifactId>
<version>1.1.1</version>
</parent>
<parent>
<groupId>org.mule.extensions</groupId>
<artifactId>mule-modules-parent</artifactId>
<version>1.1.1</version>
</parent>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-install-plugin</artifactId>
<version>2.5.2</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.2</version>
</plugin>
<plugin>
<groupId>org.jsonschema2pojo</groupId>
<artifactId>jsonschema2pojo-maven-plugin</artifactId>
<version>0.4.35</version>
<dependencies>
<dependency>
<groupId>io.github.mulesoft-consulting</groupId>
<artifactId>jsonschema2pojo-mule-annotations</artifactId>
<version>1.2.0</version>
<scope>compile</scope>
</dependency>
</dependencies>
<configuration>
<customAnnotator>org.mule.custom.annotation.utils.CustomMuleAnnotator</customAnnotator>
<sourceDirectory>${basedir}/src/main/resources/schema</sourceDirectory>
<outputDirectory>${project.build.directory}/generated-sources</outputDirectory>
<targetPackage>org.mule.extension.jsonlogger.api.pojos</targetPackage>
</configuration>
<executions>
<execution>
<goals>
<goal>generate</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-install-plugin</artifactId>
<version>2.5.2</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.2</version>
</plugin>

<distributionManagement>
<!-- Target Anypoint Organization Repository -->
<repository>
<id>Exchange2</id>
<name>Exchange2 Repository</name>
<url>https://maven.anypoint.mulesoft.com/api/v1/organizations/${project.groupId}/maven</url>
<layout>default</layout>
</repository>
</distributionManagement>
</plugins>
</build>

<dependencies>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.10.3</version>
</dependency>
<dependency>
<groupId>commons-lang</groupId>
<artifactId>commons-lang</artifactId>
<version>2.6</version>
</dependency>
<dependency>
<groupId>commons-beanutils</groupId>
<artifactId>commons-beanutils</artifactId>
<version>1.9.4</version>
</dependency>
<dependency>
<groupId>joda-time</groupId>
<artifactId>joda-time</artifactId>
<version>2.10.5</version>
</dependency>
<dependency>
<groupId>org.mule.connectors</groupId>
<artifactId>mule-jms-connector</artifactId>
<version>1.6.3</version>
<classifier>mule-plugin</classifier>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.mulesoft.connectors</groupId>
<artifactId>mule-amqp-connector</artifactId>
<version>1.6.0</version>
<classifier>mule-plugin</classifier>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.mulesoft.muleesb.modules</groupId>
<artifactId>anypoint-mq-rest-client</artifactId>
<version>3.1.0</version>
<exclusions>
<exclusion>
<artifactId>async-http-client</artifactId>
<groupId>com.ning</groupId>
</exclusion>
</exclusions>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path</artifactId>
<version>2.4.0</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.lmax</groupId>
<artifactId>disruptor</artifactId>
<version>3.4.2</version>
</dependency>
</dependencies>
<repositories>
<!-- Shared Mule Services Repository -->
<repository>
<id>Exchange2</id>
<name>Exchange2 Repository</name>
<url>https://maven.anypoint.mulesoft.com/api/v1/organizations/${project.groupId}/maven</url>
<layout>default</layout>
</repository>
</repositories>
<distributionManagement>
<repository>
<id>Exchange2</id>
<name>Exchange2 Repository</name>
<url>https://maven.anypoint.mulesoft.com/api/v1/organizations/${project.groupId}/maven</url>
<layout>default</layout>
</repository>
</distributionManagement>

<dependencies>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.10.3</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>2.14.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<repositories>
<!-- Shared Mule Services Repository -->
<repository>
<id>Exchange2</id>
<name>Exchange2 Repository</name>
<url>https://maven.anypoint.mulesoft.com/api/v1/organizations/${project.groupId}/maven</url>
<layout>default</layout>
</repository>
</repositories>
</project>
94 changes: 0 additions & 94 deletions json-logger/pom.xml.original
View file
Open in desktop

This file was deleted.

20 changes: 20 additions & 0 deletions json-logger/src/main/java/org/mule/extension/jsonlogger/Constants.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
package org.mule.extension.jsonlogger;

class Constants {
static final String APPLICATION_NAME = "applicationName";
static final String APPLICATION_VERSION = "applicationVersion";
static final String CONTENT = "content";
static final String CORRELATION_ID = "correlationId";
static final String ENVIRONMENT = "environment";
static final String LOCATION_INFO = "locationInfo";
static final String ROOT_CONTAINER = "rootContainer";
static final String FILE_NAME = "fileName";
static final String LINE_NUMBER = "lineNumber";
static final String TIMSTAMP = "timestamp";
static final String MESSAGE = "message";
static final String PRIORITY = "priority";
static final String TRACE_POINT = "tracePoint";
static final String ELAPSED = "elapsed";
static final String START_TIME = "startTime";
static final String END_TIME = "endTime";
}
13 changes: 13 additions & 0 deletions json-logger/src/main/java/org/mule/extension/jsonlogger/JsonLoggerExtension.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
package org.mule.extension.jsonlogger;

import org.mule.extension.jsonlogger.config.JsonLoggerConfig;
import org.mule.runtime.extension.api.annotation.Configurations;
import org.mule.runtime.extension.api.annotation.Extension;
import org.mule.runtime.extension.api.annotation.dsl.xml.Xml;

@Xml(prefix = "json-logger")
@Extension(name = "JSON Logger")
@Configurations(JsonLoggerConfig.class)
public class JsonLoggerExtension {

}
Loading