Skip to content

feat: replaced pysnmplib with lextudio's pysnmp v7.1.8 #1276

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
gmaiyani-crest merged 66 commits into from
Dec 2, 2025
Merged

feat: replaced pysnmplib with lextudio's pysnmp v7.1.8 #1276

gmaiyani-crest merged 66 commits into from
Dec 2, 2025

Conversation

@gmaiyani-crest
Copy link

@gmaiyani-crest gmaiyani-crest commented Oct 24, 2025 •
edited
Loading

Description

Adding support of python v3.13 and migrating from pysnmplib to lextudio's pysnmp v7.1.8

  • Replaced pysnmplib with lextudio’s pysmp v7.1.8
  • Replaced pysnmp-pyasn1 to lextudio's pyasn1
  • Replaced pysnmp-pysmi to lextudio's pysmi
  • Included pysnmpcrypto (maintained by lextudio's) for stronger crypto algorithms with pysnmp v6+
  • Made SC4SNMP compatible with Python 3.13
  • Updated SNMP walk implementation from bulkCmd to custom multi_bulk_walk_cmd
  • Introduced deploymentUser in values.yaml to make the sc4snmp entirely compatible with lextudio's pysnmp.
  • Added DEPLOYMENT_USER variable for Docker Compose deployment.
  • Introduces multi_bulk_walk_cmd to walk multiple OID trees simultaneously with lexicographic mode control per varbind.

Notes

Added custom multi_bulk_walk_cmd function for Multi-Varbind SNMP Walking.

Why?

  • bulk_walk_cmd supports only a single varbind at a time and already includes lexicographic mode handling.
    However, to achieve the same behavior for multiple varbinds (from different OID subtrees), each varbind must be walked separately in sequence.
  • Current bulk_cmd of lextuido's pysnmp does not support lexicographic option as a result we can not use bulk_cmd of lextudio's pysnmp for the same purpose.
  • The new custom multi_bulk_walk_cmd addresses this by allowing multiple varbinds to be walked in parallel within a single session, maintaining lexicographic consistency while significantly reducing network roundtrips and improving overall efficiency as compare to previous bulkCmd.
  • Feature request PR for adding multi_bulk_walk_cmd at Lextudio — raised here: [PR-LINK]

Key Changes

  • Added multi_bulk_walk_cmd function for parallel walking of multiple OID subtrees
  • Implemented lexicographic mode control for each varbind
  • Restored etingof's bulkCmd multi-varbind with lexicographic capability with modern async/await patterns
  • Implemented precise maxRows handling that counts actual OIDs (not iterations) with automatic truncation
  • Added per-varbind completion tracking for independent state management

Type of change

Please delete options that are not relevant.

  • Dependency update
  • New feature
  • Refactor/improvement
  • This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

Checklist

  • My commit message is conventional
  • I have run pre-commit on all files before creating the PR
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have checked my code and corrected any misspellings

ikheifets-splunk and others added 30 commits June 10, 2025 08:42
@ikheifets-splunk @omrozowicz-splunk @ajasnosz
feat: mTLS for Splunk 10 (#1197)
e8cb2a4 
* feat: mTLS support for Splunk 10

* docs: fix typo

---------

Co-authored-by: Olga <[email protected]>
Co-authored-by: ajasnosz <[email protected]>
@srv-rr-github-token
chore(release): 1.13.0-beta.1
e15d414 
# [1.13.0-beta.1](v1.12.3...v1.13.0-beta.1) (2025-06-10)

### Features

* mTLS for Splunk 10 ([#1197](#1197)) ([e8cb2a4](e8cb2a4))
@renovate
chore(deps): update docker/login-action action to v3.4.0 (#1166)
0324bbb 
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
@ajasnosz
fix: python and dependency update (#1200)
dd39338
@srv-rr-github-token
chore(release): 1.13.0-beta.2
463e4ea 
# [1.13.0-beta.2](v1.13.0-beta.1...v1.13.0-beta.2) (2025-06-10)

### Bug Fixes

* python and dependency update ([#1200](#1200)) ([dd39338](dd39338))
@omrozowicz-splunk
chore: add missing sudo in integration test script (#1207)
c911e7a
@omrozowicz-splunk
chore: update microk8s (#1206)
b9a0294 
* chore: update microk8s
* chore: add changelog
@renovate @omrozowicz-splunk
chore(deps): update dependency urllib3 to v2 [security] (#1205)
c3dd6c3 
* chore(deps): update dependency urllib3 to v2 [security]

* test: test [run-int-tests]

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: omrozowicz-splunk <[email protected]>
@ajasnosz
feat: add discovery for docker (#1208)
2559289 
* feat: add discovery for docker

* chore: update replica count

* chore: update replica count
@cwadhwani-splunk
feat: Added deployment for k8s (#1204)
e1f3ffb 
* feat: Added deployment for k8s

* Changed the queue name from autodiscover to discovery
@omrozowicz-splunk
fix: replace - with _ for metrics (#1213)
cd44b56 
* fix: add option to change "-" to an "_" to be compatible with metric schema in Splunk

* fix: doc fix and docker-compose change

* fix: add docs

* chore: pre-commit run

* doc: refine

* doc: address PR comments, update CHANGELOG
@srv-rr-github-token
chore(release): 1.13.0-beta.3
582a553 
# [1.13.0-beta.3](v1.13.0-beta.2...v1.13.0-beta.3) (2025-07-09)

### Bug Fixes

* replace - with _ for metrics ([#1213](#1213)) ([cd44b56](cd44b56))
@ajasnosz
feat: add partial walk (#1215)
0e1f32f
@srv-rr-github-token
chore(release): 1.13.0-beta.4
9219959 
# [1.13.0-beta.4](v1.13.0-beta.3...v1.13.0-beta.4) (2025-07-22)

### Features

* add partial walk ([#1215](#1215)) ([0e1f32f](0e1f32f))
@omrozowicz-splunk
test: parallelised integration tests (#1220)
e98c98a 
* test: add one more worker

* chore: add parallelism to docker-compose tests

* chore: move some polling tests to worker
@omrozowicz-splunk
fix: problem with casting strings of scientific notation (#1219)
80818aa 
* fix: problems with interpreting scientific value as float

* fix: unnecessary import

* fix: run pre-commit

* fix: check regex only if value is an instance of str

* fix: wrong function

* test: delete unnecessary test case

* test: rename test file

* chore: run pre-commit

* chore: update CHANGELOG
@srv-rr-github-token
chore(release): 1.13.0-beta.5
859c84d 
# [1.13.0-beta.5](v1.13.0-beta.4...v1.13.0-beta.5) (2025-07-30)

### Bug Fixes

* problem with casting strings of scientific notation ([#1219](#1219)) ([80818aa](80818aa))
@ajasnosz
fix: add ipv6 flag for worker poller (#1218)
57d44ae
@srv-rr-github-token
chore(release): 1.13.0-beta.6
41f237c 
# [1.13.0-beta.6](v1.13.0-beta.5...v1.13.0-beta.6) (2025-07-31)

### Bug Fixes

* add ipv6 flag for worker poller ([#1218](#1218)) ([57d44ae](57d44ae))
@renovate
chore(deps): update dependency mkdocs-material to v9.6.16 (#1211)
be59765 
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
@cwadhwani-splunk
feat: Feat/autodiscovery application code (#1217)
0600650 
* feat: initial commit with application code

* Added logic for mounting the final output csv file

* feat: addressed review comments

* chore: removed the init container and handled the volume mounting without it

* feat: added support for scaling discovery worker pods

* chore: Updated port validation

* feat: Added support for snmp version 3 in docker and handled volume mounting

* fix: Corrected the discovery paths in environment variable and removed unnecessary variables
@omrozowicz-splunk
chore: fix license accept issue in integration tests (#1227)
7b18a91
@cwadhwani-splunk
feat: added configurable parameter for SNMP request's retry and timeo… (
48641c8 
#1224)

* feat: added configurable parameter for SNMP request's retry and timeout. Handled tasktimeout configurable parameter for discovery

* fix: handled race condition when discovery tasks tries to update the output csv file concurrently

* chore: Executed make render command with the latest yaml changes

* chore: removed --min-rate as nmap dynamically detects a suitable rate at which packets should be sent

* chore: incorporated the pr review changes
@omrozowicz-splunk
fix: logging (#1223)
a98987c 
* fix: disable mongo logs, fix traps logging

* fix: run pre-commit, fix conflicts

* fix: run pre-commit

* fix: make mongo logging configurable

* fix: make mongo logging configurable

* fix: add docs

* fix: fix traps helm template 

* fix: add CHANGELOG
@srv-rr-github-token
chore(release): 1.13.0-beta.7
0da4200 
# [1.13.0-beta.7](v1.13.0-beta.6...v1.13.0-beta.7) (2025-08-13)

### Bug Fixes

* logging ([#1223](#1223)) ([a98987c](a98987c))
@omrozowicz-splunk
fix: update dependencies (#1229)
fcf8805
@srv-rr-github-token
chore(release): 1.13.0-beta.8
c89dd17 
# [1.13.0-beta.8](v1.13.0-beta.7...v1.13.0-beta.8) (2025-08-13)

### Bug Fixes

* update dependencies ([#1229](#1229)) ([fcf8805](fcf8805))
@cwadhwani-splunk
docs: Added doc changes for discovery feature (#1228)
71108d7 
* docs: Added doc changes for discovery feature

* docs: spell checks

* chore: resolved pr review comments
@cwadhwani-splunk
chore: removed expires from the shared_task decorator
289d48a
@omrozowicz-splunk @ajasnosz
fix: traps add engine contextId (#1230)
3dff9cd 
* fix: add decode method security engine ID

* chore: add missing import

* fix: unit tests

* fix: run pre-commit

* fix: update docker env file, docs and changelog

* fix: update method description

---------

Co-authored-by: ajasnosz <[email protected]>
@gmaiyani-crest gmaiyani-crest changed the title feat: replaced pysnmplib with lextudio's pysnmp v7.1.0 feat: replaced pysnmplib with lextudio's pysnmp v7.1.8 Oct 30, 2025
@gmaiyani-crest gmaiyani-crest force-pushed the feat/migrate-pysnmplib-to-lextudio-pysnmp-v7.1.0 branch from 62784f2 to dcd181c Compare November 10, 2025 09:06
@gmaiyani-crest
merge: merged branch 'feat/migrate-pysnmplib-to-lextudio-pysnmp-v7.1.…
d248798 
…0' of github.com:splunk/splunk-connect-for-snmp into feat/migrate-autodiscovery
@gmaiyani-crest
merge: pull of branch 'feat/autodiscovery' of github.com:splunk/splun…
0a38147 
…k-connect-for-snmp into feat/migrate-autodiscovery
@gmaiyani-crest
merge: merged branch 'feat/autodiscovery' of github.com:splunk/splunk…
90c256c 
…-connect-for-snmp into feat/migrate-autodiscovery
@gmaiyani-crest
chore: migrate auto-discovery feature to make compatible with lextuio…
bfab98b 
…'s pysnmp and python 3.13

- Migrate auto-discovery feature to make compitable with lextuio's pysnmp and python 3.13
- Replace the thread-pool executor with semaphore asyncio for concurrency control
@gmaiyani-crest
merge: pull latest changes from the develop branch
d719fa7
@gmaiyani-crest gmaiyani-crest merged commit e4afa46 into feat/support-of-python-3.13-and-replaced-pysnmplib-to-lextudio-pysnmp Dec 2, 2025
2 checks passed
@gmaiyani-crest gmaiyani-crest deleted the feat/migrate-pysnmplib-to-lextudio-pysnmp-v7.1.0 branch December 2, 2025 12:32
@github-actions github-actions bot locked and limited conversation to collaborators Dec 2, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@omrozowicz-splunk omrozowicz-splunk omrozowicz-splunk approved these changes

@ajasnosz ajasnosz ajasnosz approved these changes

Assignees

@gmaiyani-crest gmaiyani-crest

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@gmaiyani-crest @omrozowicz-splunk @ajasnosz @ikheifets-splunk @srv-rr-github-token @cwadhwani-splunk