Skip to content

Latest commit

 

History

History
248 lines (183 loc) · 8.97 KB

README.md

File metadata and controls

248 lines (183 loc) · 8.97 KB

🌐 中文简体 | English

ip2region python query client

Version Compatibility

This implementation is compatible with Python >= 3.7

Usage

Install py-ip2region

pip3 install py-ip2region

About Query API

The prototype of the Query API is:

# Query via string IP or binary IP (bytes type) parsed by util.parse_ip
search(ip: str | bytes)

An exception will be thrown if the query fails. If successful, the region information in string format will be returned. If the specified IP cannot be found, an empty string "" will be returned.

About IPv4 and IPv6

This xdb query client implementation supports both IPv4 and IPv6 queries. Use it as follows:

import ip2region.util as util

# For IPv4: Set xdb path to the v4 xdb file, and specify the IP version as util.IPv4
db_path = "../../data/ip2region_v4.xdb"  # Or your ipv4 xdb path
version = util.IPv4

# For IPv6: Set xdb path to the v6 xdb file, and specify the IP version as util.IPv6
db_path = "../../data/ip2region_v6.xdb"  # Or your ipv6 xdb path
version = util.IPv6

# The IP version of the xdb specified by db_path must match the version specified by version, otherwise an error will occur during query execution
# Note: The following demonstrations directly use the db_path and version variables

File Verification

It is recommended that you proactively verify the applicability of the xdb file. Future new features may cause the current Searcher version to be incompatible with the xdb file you are using; verification can prevent unpredictable errors during runtime. You do not need to verify every time. For example, perform verification when the service starts or manually call the command to confirm version matching. Do not run verification every time a Searcher is created, as this will affect query response speed, especially in high-concurrency scenarios.

import ip2region.util as util

try:
    util.verify_from_file(db_path)
except Exception e:
    # Applicability verification failed!!!
    # The current query client implementation is not applicable for the xdb file specified by db_path.
    # You should stop the service and use a suitable xdb file or upgrade to a Searcher implementation suitable for db_path.
    print(f"binding is not applicable for xdb file '{db_path}': {str(e)}")
    return

# Verification passed, the current Searcher can be safely used for query operations on the xdb pointed to by db_path

File-Only Query

import ip2region.util as util
import ip2region.searcher as xdb

# 1. Use the version and db_path mentioned above to create a file-only query object
try:
    searcher = xdb.new_with_file_only(version, db_path)
except Exception as e:
    print(f"failed to new_with_file_only: {str(e)}")
    return


# 2. Query, it supports both IPv4 and IPv6 addresses
ip = "1.2.3.4"
# ip = "240e:3b7:3272:d8d0:db09:c067:8d59:539e"  // IPv6
try:
    region = searcher.search(ip)
    print(f"search({ip}): {{region: {region}, io_count: {searcher.get_io_count()}}}")
except Exception as e:
    print(f"failed to search: {str(e)}")

# 3. Close resources
searcher.close()

# Note: Each thread needs to create an independent Searcher object

Caching VectorIndex

We can pre-load the VectorIndex data from the xdb file and cache it globally. Using the global VectorIndex cache every time a Searcher object is created can reduce a fixed IO operation, thereby accelerating queries and reducing IO pressure.

import ip2region.util as util
import ip2region.searcher as xdb

# 1. Pre-load VectorIndex cache from db_path and use this data as a global variable for subsequent repeated use.
try:
    v_index = util.load_vector_index_from_file(db_path)
except Exception as e:
    print(f"failed to load vector index from {db_path}: {str(e)}")
    return

# 2. Use the global v_index to create a query object with VectorIndex cache.
try:
    searcher = xdb.new_with_vector_index(version, db_path, v_index)
except Exception as e:
    print(f"failed to new_with_vector_index: {str(e)}")
    return


# 3. Query; the interface is the same for both IPv4 and IPv6 addresses
ip = "1.2.3.4"
# ip = "240e:3b7:3272:d8d0:db09:c067:8d59:539e"  // IPv6
try:
    region = searcher.search(ip)
    print(f"search({ip}): {{region: {region}, io_count: {searcher.get_io_count()}}}")
except Exception as e:
    print(f"failed to search: {str(e)}");

# 4. Close resources
searcher.close()

# Note: Each thread needs to create an independent Searcher object, but they all share the global read-only v_index cache.

Caching the Entire xdb File

We can also pre-load the data of the entire xdb file into memory and create a query object based on this data to achieve fully memory-based queries, similar to the previous memory search.

import ip2region.util as util
import ip2region.searcher as xdb

# 1. Load the entire xdb into memory from db_path.
try:
    c_buffer = util.load_content_from_file(db_path)
except Exception as e:
    print(f"failed to load content from {db_path}: {str(e)}")
    return

# 2. Use the c_buffer mentioned above to create a fully memory-based query object.
try:
    searcher = xdb.new_with_buffer(version, c_buffer)
except Exception e:
    print(f"failed to new_with_buffer: {str(e)}")
    return

# 3. Query; the interface is the same for both IPv4 and IPv6 addresses
ip = "1.2.3.4"
# ip = "240e:3b7:3272:d8d0:db09:c067:8d59:539e"  # IPv6
try:
    region = searcher.search(ip)
    print(f"search({ip}): {{region: {region}, io_count: 0}}")
except Exception as e:
    print(f"failed to search: {str(e)}")
        
# 4. Close resources - This searcher object can be safely used for concurrency; close the searcher only when the entire service is going to shut down
# searcher.close()

# Note: For concurrent use, query objects created with the entire xdb data cache can be safely used for concurrency, meaning you can make this searcher object a global object for cross-thread access.

Query Test

You can test queries via the python3 search_test.py command:

➜  python git:(fr_python_ipv6) ✗ python3 search_test.py 
usage: python search_test.py [command option]

ip2region search test script

options:
  -h, --help            show this help message and exit
  --db DB               ip2region binary xdb file path
  --cache-policy CACHE_POLICY
                        cache policy: file/vectorIndex/content, default: vectorIndex

For example: Use the default data/ip2region_v4.xdb file for IPv4 query testing:

➜  python git:(fr_python_ipv6) ✗ python3 search_test.py --db=../../data/ip2region_v4.xdb                       
ip2region xdb searcher test program
source xdb: ../../data/ip2region_v4.xdb (IPv4, vectorIndex)
type 'quit' to exit
ip2region>> 1.2.3.4
{region: Australia|Queensland|Brisbane|0|AU, ioCount: 5, took: 188 μs}
ip2region>> 113.118.113.77
{region: 中国|广东省|深圳市|电信|CN, ioCount: 2, took: 143 μs}

For example: Use the default data/ip2region_v6.xdb file for IPv6 query testing:

➜  python git:(fr_python_ipv6) ✗ python3 search_test.py --db=../../data/ip2region_v6.xdb 
ip2region xdb searcher test program
source xdb: ../../data/ip2region_v6.xdb (IPv6, vectorIndex)
type 'quit' to exit
ip2region>> ::
{region: , ioCount: 1, took: 166 μs}
ip2region>> 240e:3b7:3272:d8d0:db09:c067:8d59:539e
{region: 中国|广东省|深圳市|电信|CN, ioCount: 8, took: 197 μs}
ip2region>> 2604:a840:3::a04d
{region: United States|California|San Jose|xTom|US, ioCount: 13, took: 240 μs}

Input an IP to perform a query test. You can also set cache-policy to file/vectorIndex/content respectively to test the effects of the three different cache implementations.

bench Test

Bench testing can be performed via the python3 bench_test.py command, which ensures the xdb file is error-free and allows for performance evaluation:

➜  python git:(fr_python_ipv6) ✗ python3 bench_test.py                                                                                         
usage: python bench_test.py [command option]

ip2region bench test script

options:
  -h, --help            show this help message and exit
  --db DB               ip2region binary xdb file path
  --src SRC             source ip text file path
  --cache-policy CACHE_POLICY
                        cache policy: file/vectorIndex/content, default: vectorIndex

For example: Perform IPv4 bench testing via the default data/ip2region_v4.xdb and data/ipv4_source.txt files:

python3 bench_test.py --db=../../data/ip2region_v4.xdb --src=../../data/ipv4_source.txt

For example: Perform IPv6 bench testing via the default data/ip2region_v6.xdb and data/ipv6_source.txt files:

python3 bench_test.py --db=../../data/ip2region_v6.xdb --src=../../data/ipv6_source.txt

You can test the effects of the three different cache implementations by setting cache-policy to file/vectorIndex/content respectively. @Note: Ensure the src file used for bench is the same source file used to generate the corresponding xdb file.