Skip to content

Commit 9358f18

Browse files
eschultinkdependabot[bot]Copilotaperez-worklytics
authored
v0.5.5 (#953)
* update release refs to rc-v0.5.5 * misc improvements: java24, async mem (#941) * in async case, double mem size to avoid problems * java 24 support * set maven version in github-action * fix action * clean-up docs * Bump form-data from 2.5.1 to 2.5.5 in /tools/psoxy-test (#942) Bumps [form-data](https://github.com/form-data/form-data) from 2.5.1 to 2.5.5. - [Release notes](https://github.com/form-data/form-data/releases) - [Changelog](https://github.com/form-data/form-data/blob/v2.5.5/CHANGELOG.md) - [Commits](form-data/form-data@v2.5.1...v2.5.5) --- updated-dependencies: - dependency-name: form-data dependency-version: 2.5.5 dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix webhook collectors (#940) * add a truncate to avoid max length issues * attempt to fix webhook tf apply circular loop * fix extra values() call; clarify why these are sorted * cleanup webhook collector notes in dev * doc webhoook mode * top-level README updates * more doc improvements * Update infra/modules/aws-webhook-collector/main.tf Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: Copilot <[email protected]> * release tooling fixes (#943) * clean top README * update doc instructions * release publish fixes * fix build tool for release artifacts * json schema filter / parameter schema `anyOf` (replacing `oneOf`/`or`) (#944) * rename oneOf --> anyOf * for completeness, same thing for ParameterSchema * Update java/gateway-core/src/main/java/com/avaulta/gateway/rules/JsonSchemaFilterUtils.java Co-authored-by: aperez-worklytics <[email protected]> * Update java/gateway-core/src/main/java/com/avaulta/gateway/rules/JsonSchemaFilterUtils.java Co-authored-by: aperez-worklytics <[email protected]> --------- Co-authored-by: aperez-worklytics <[email protected]> * bom robustness (#945) * bring in BOM file input fix * cleanup after cherry-pick * cleanup BOM parsing * fix test * fix test * Adding test * Check body content * Better test * Using mime type * Adding contains * Adding test * fix concurrency (async case) (#946) * attempt to fix concurrency; some methods in tests giving issues * use plain Future<InputStream> * Revert "use plain Future<InputStream>" This reverts commit afb9f7a. * recover a ProcessedStream-based implementation * add some comments * First test * Checking endpoints and sanitized responses * Chat GPT spec * Drop hyphen * Exposed variable * Updated files * Updated YAML * Lint and format * Basic readme * Updated instructions * Update java/core/src/main/java/co/worklytics/psoxy/rules/chatgpt/PrebuiltSanitizerRules.java Co-authored-by: Copilot <[email protected]> * Update infra/modules/worklytics-connector-specs/variables.tf Co-authored-by: Copilot <[email protected]> * Update infra/modules/worklytics-connectors/variables.tf Co-authored-by: Copilot <[email protected]> * Fix name * Updated YAML * Typos * Update infra/modules/worklytics-connector-specs/main.tf Co-authored-by: Copilot <[email protected]> * Typo in line * Better check * Rules with text digest * Renamed to enterprise * Renamed to enterprise * Fix typo * Fix source * Using access token * Updated test * support Webhook Collection in gcp-hosted deployments (#948) * wip of GCP webhook collection, java only * initial version of gcp tf infra * fix api data request for google case (possibly more??) * improve doc of gcp prereqs * numerous fixes, but still wip * various fixes, but auth still failing * fix weird key-ring names * java fixes to make gcp webhook collection work OK * remove some stray stuff * cron scheduler job * fix test * fix, add logging * tf style cleanup * fix attribute parsing * cleanup merge handler, fix lack of newlines * fix AUTH_ISSUER access issues * test file for GcpWebhookCollectionHandler * avoid writing empty output files * support configurable timeouts, batch sizes in java side * doc some gcp webhook collector constants * doc gcp roles * fix test * fix test * disable cursor test * Update java/impl/gcp/src/main/java/co/worklytics/psoxy/GcpWebhookCollectionHandler.java Co-authored-by: Copilot <[email protected]> * Update java/core/src/main/java/co/worklytics/psoxy/gateway/auth/Base64KeyClient.java Co-authored-by: Copilot <[email protected]> * fix aws webhooks * CR feedback * unused init * comment on tf module design --------- Co-authored-by: Copilot <[email protected]> * update release refs to v0.5.5 (#950) * align GCP webhook interface to AWS (#952) * align gcp webhook example to aws * try() in examples --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Copilot <[email protected]> Co-authored-by: aperez-worklytics <[email protected]> Co-authored-by: andres <[email protected]>
1 parent 926bdf1 commit 9358f18

File tree

111 files changed

+4756
-1083
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

111 files changed

+4756
-1083
lines changed

.github/workflows/build-java.yaml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,11 @@ on:
1616
type: string
1717
description: java distribution to use # see https://github.com/actions/setup-java#supported-distributions
1818
default: corretto
19+
maven-version:
20+
required: false
21+
type: string
22+
description: maven version to use # see https://github.com/s4u/setup-maven-action#supported-version-syntax
23+
default: '3.9.10' # 3.9.10+ is required for Java 24; in theory, we support earlier shoudl we test?
1924

2025
jobs:
2126
ci_java:
@@ -29,6 +34,11 @@ jobs:
2934
java-version: ${{ inputs.java-version }}
3035
# https://github.com/actions/setup-java#supported-distributions
3136
distribution: ${{ inputs.java-distribution }}
37+
- name: Setup Maven Action
38+
39+
with:
40+
java-version: ${{ inputs.java-version }}
41+
maven-version: ${{ inputs.maven-version }}
3242
- name: Cache Maven packages
3343
uses: actions/cache@v3
3444
with:

.github/workflows/ci-java-all.yaml

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ jobs:
1919
uses: ./.github/workflows/build-java.yaml
2020
with:
2121
java-version: '17'
22+
maven-version: '3.6.3' # just for interests sake
2223

2324
# Java 21 - released 19 Sept 2023, supported until Sept 2028 (LTS)
2425
ci_java21:
@@ -32,10 +33,8 @@ jobs:
3233
with:
3334
java-version: '23'
3435

35-
# Java 24 disabled, pending Maven releasing 3.9.10 or 4.x version that supports Java 24
36-
# specific issue is https://issues.apache.org/jira/browse/MNG-8248
37-
# # Java 24 - released 18 March 2025, supported until Sept 2025
38-
# ci_java24:
39-
# uses: ./.github/workflows/build-java.yaml
40-
# with:
41-
# java-version: '24'
36+
# Java 24 - released 18 March 2025, supported until Sept 2025
37+
ci_java24:
38+
uses: ./.github/workflows/build-java.yaml
39+
with:
40+
java-version: '24'

README.md

Lines changed: 10 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -6,31 +6,18 @@
66

77
A serverless, pseudonymizing, DLP layer between Worklytics and the REST API of your data sources.
88

9-
Psoxy replaces PII in your organization's data with hash tokens to enable Worklytics's analysis to
10-
be performed on anonymized data which we cannot map back to any identifiable individual.
9+
Psoxy replaces PII in your organization's data with hash tokens to enable Worklytics's analysis to be performed on anonymized data which we cannot map back to any identifiable individual.
1110

12-
Psoxy is a pseudonymization service that acts as a Security / Compliance layer, which you can deploy
13-
between your data sources (SaaS tool APIs, Cloud storage buckets, etc) and the tools that need to
14-
access those sources.
11+
Psoxy is a pseudonymization service that acts as a Security / Compliance layer, which you can deploy between your data sources (SaaS tool APIs, Cloud storage buckets, etc) and the tools that need to access those sources.
1512

16-
Psoxy ensures more secure, granular data access than direct connections between your tools will
17-
offer - and enforces access rules to fulfill your Compliance requirements.
13+
Psoxy ensures more secure, granular data access than direct connections between your tools will offer - and enforces access rules to fulfill your Compliance requirements.
1814

19-
Psoxy functions as API-level Data Loss Prevention layer (DLP), by blocking sensitive fields / values
20-
/ endpoints that would otherwise be exposed when you connect a data sources API to a 3rd party
21-
service. It can ensure that data which would otherwise be exposed to a 3rd party service, due to
22-
granularity of source API models/permissions, is not accessed or transfered to the service.
15+
Psoxy functions as API-level Data Loss Prevention layer (DLP), by blocking sensitive fields / values / endpoints that would otherwise be exposed when you connect a data sources API to a 3rd party service. It can ensure that data which would otherwise be exposed to a 3rd party service, due to granularity of source API models/permissions, is not accessed or transferred to the service.
2316

2417
Objectives:
25-
- **serverless** - we strive to minimize the moving pieces required to run psoxy at scale, keeping
26-
your attack surface small and operational complexity low. Furthermore, we define
27-
infrastructure-as-code to ease setup.
28-
- **transparent** - psoxy's source code is available to customers, to facilitate code review
29-
and white box penetration testing.
30-
- **simple** - psoxy's functionality will focus on performing secure authentication with the 3rd
31-
party API and then perform minimal transformation on the response (pseudonymization, field
32-
redaction) to ease code review and auditing of its behavior.
33-
18+
- **serverless** - we strive to minimize the moving pieces required to run psoxy at scale, keeping your attack surface small and operational complexity low. Furthermore, we define infrastructure-as-code to ease setup.
19+
- **transparent** - psoxy's source code is available to customers, to facilitate code review and white box penetration testing.
20+
- **simple** - psoxy's functionality will focus on performing secure authentication with the 3rd party API and then perform minimal transformation on the response (pseudonymization, field redaction) to ease code review and auditing of its behavior.
3421

3522
## Documentation
3623

@@ -40,7 +27,6 @@ For development purposes, latest docs are also accessible at GitHub [docs/](./do
4027

4128
## Support
4229

43-
Psoxy is maintained by Worklytics, Co. Support as well as professional services to assist with
44-
configuration and customization are available. Please contact
45-
[[email protected]](mailto:[email protected]) for more information or visit
46-
[www.worklytics.co](https://www.worklytics.co).
30+
Psoxy is maintained by Worklytics, Co.
31+
32+
Support, as well as professional services, to assist with configuration and customization are available. Please contact [[email protected]](mailto:[email protected]) for more information or visit [www.worklytics.co](https://www.worklytics.co).

docs/README.md

Lines changed: 9 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -47,26 +47,22 @@ But Psoxy ensures Worklytics only sees:
4747
}
4848
```
4949

50-
These pseudonyms leverage SHA-256 hashing / AES encryption, with salt/keys that are known only to
51-
your organization and never transferred to Worklytics.
50+
These pseudonyms leverage SHA-256 hashing / AES encryption, with salt/keys that are known only to your organization and never transferred to Worklytics.
5251

53-
Psoxy enforces that Worklytics can only access API endpoints you've configured ([principle of least
54-
privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege)) using HTTP methods you allow (eg, limit to `GET` to enforce read-only for RESTful APIs).
52+
Psoxy enforces that Worklytics can only access API endpoints you've configured ([principle of least privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege)) using HTTP methods you allow (eg, limit to `GET` to enforce read-only for RESTful APIs).
5553

56-
For data sources APIs which require keys/secrets for authentication, such values remain stored in
57-
your premises and are never accessible to Worklytics.
54+
For data sources APIs which require keys/secrets for authentication, such values remain stored in your premises and are never accessible to Worklytics.
5855

59-
You authorize your Worklytics tenant to access your proxy instance(s) via the IAM platform of your
60-
cloud host.
56+
You authorize your Worklytics tenant to access your proxy instance(s) via the IAM platform of your cloud host.
6157

6258
Worklytics authenticates your tenant with your cloud host via [Workload Identity Federation](https://cloud.google.com/iam/docs/workload-identity-federation). This eliminates the need for any secrets to be exchanged between your organization and Worklytics, or the use any API keys/certificates for Worklytics which you would need to rotate.
6359

6460
See also: [API Data Sanitization](configuration/api-data-sanitization.md)
6561

6662
## Supported Data Sources
67-
As of March 2023, the following sources can be connected to Worklytics via psoxy.
63+
As of July 2025, the following sources can be connected to Worklytics via psoxy.
6864

69-
Note: Some sources require specific licenses to transfer data via the APIs/endpoints used by Worklytics, or impose some per API request costs for such transfers.
65+
Note: Some sources require specific licenses to transfer data via the APIs/endpoints used by Worklytics, or impose some per API request costs/rate limits for such transfers. Inclusion of the source in the list below does not represent or warrant retrieval of your data using Psoxy from the source via our provided connectors.
7066

7167
### Google Workspace (formerly GSuite)
7268

@@ -114,7 +110,7 @@ NOTE: usage of the Microsoft Teams APIs may be billable, depending on your Micro
114110

115111
See details: [sources/microsoft-365/README.md](sources/microsoft-365/README.md)
116112

117-
### Github
113+
### GitHub
118114

119115
Check the documentation to use the right permissions and the right authentication flow per connector.
120116

@@ -214,11 +210,10 @@ You will need all the following in your deployment environment (eg, your laptop)
214210
| Tool | Version | Test Command |
215211
|----------------------------------------------|----------------------|-----------------------|
216212
| [git](https://git-scm.com/) | 2.17+ | `git --version` |
217-
| [Maven](https://maven.apache.org/) | 3.6+ | `mvn -v` |
218-
| [Java JDK 17+](https://openjdk.org/install/) | 17, 21 (see notes) | `mvn -v \| grep Java` |
213+
| [Maven](https://maven.apache.org/) | 3.6+ ; 3.9.10+ required for java 24 | `mvn -v` |
214+
| [Java JDK 17+](https://openjdk.org/install/) | 17, 21, 24 (see notes) | `mvn -v \| grep Java` |
219215
| [Terraform](https://www.terraform.io/) | 1.6+, < 2.0 | `terraform version` |
220216

221-
NOTE: as of Apr 8, 2024, although Java 24 has been released Maven 3.9.9 is not compatible with it. Maven has fixed this, but has yet to release a version 3.9.10 or 4.0.x with the fix. Until then, we don't officially support Java 24.
222217

223218
NOTE: we will support Java versions for duration of official support windows, in particular the LTS versions. Minor versions, such as 18-20, 22-23 which are out of official support, may work but are not routinely tested.
224219

0 commit comments

Comments
 (0)