archive-gh-me/ip2region
1
0
Fork 0
mirror of https://github.com/lionsoul2014/ip2region.git synced 2025-12-08 19:25:22 +00:00
Code Issues Projects Releases Wiki Activity

new javascript binding is Ready with IPv4 and IPv6 supported

Browse Source
This commit is contained in:
lion 2025-10-15 11:27:10 +08:00
parent f03d33e5fe
commit fbadc7cca0
27 changed files with 17 additions and 7677 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

29
ReadMe.md
View File

@ -33,21 +33,20 @@
API 介绍，使用文档和测试程序请参考对应 `searcher` 查询客户端下的 ReadMe 介绍，全部查询 binding 实现情况如下：
| 编程语言 | 描述 | IPv4 支持 | IPv6 支持 | 贡献者 |
|:---------------------------------|:----------------------------|:-------------------|:------------------ |:--------------------------------------------------|
| [Golang](binding/golang) | golang xdb 查询客户端 | :white_check_mark: | :white_check_mark: | [Lion](https://github.com/lionsoul2014) |
| [PHP](binding/php) | php xdb 查询客户端 | :white_check_mark: | :white_check_mark: | [Lion](https://github.com/lionsoul2014) |
| [Java](binding/java) | java xdb 查询客户端 | :white_check_mark: | :white_check_mark: | [Lion](https://github.com/lionsoul2014) |
| [C](binding/c) | POSIX C xdb 查询客户端 | :white_check_mark: | :white_check_mark: | [Lion](https://github.com/lionsoul2014) |
| [Lua_c](binding/lua_c) | lua c 扩展 xdb 查询客户端 | :white_check_mark: | :white_check_mark: | [Lion](https://github.com/lionsoul2014) |
| [Lua](binding/lua) | lua xdb 查询客户端 | :white_check_mark: | :x: | [Lion](https://github.com/lionsoul2014) |
| [Rust](binding/rust) | rust xdb 查询客户端 | :white_check_mark: | :white_check_mark: | [gongzhengyang](https://github.com/gongzhengyang) |
| [Python](binding/python) | python xdb 查询客户端 | :white_check_mark: | :x: | [厉害的花花](https://github.com/luckydog6132) |
| [Nodejs](binding/nodejs) | nodejs xdb 查询客户端 | :white_check_mark: | :x: | [Wu Jian Ping](https://github.com/wujjpp) |
| [Csharp](binding/csharp) | csharp xdb 查询客户端 | :white_check_mark: | :x: | [Alen Lee](https://github.com/malus2077) |
| [Erlang](binding/erlang) | erlang xdb 查询客户端 | :white_check_mark: | :x: | [malou](https://github.com/malou996) |
| [Nginx](binding/nginx) | nginx 扩展 xdb 查询客户端 | :white_check_mark: | :x: | [Wu Jian Ping](https://github.com/wujjpp) |
| [C++](binding/cpp) | C++ xdb 查询客户端 | :white_check_mark: | :x: | [Yunbin Liu](https://github.com/liuyunbin) |
| [Typescript](binding/typescript) | Typescript xdb 查询客户端 | :white_check_mark: | :x: | [Alan Lee](https://github.com/malus2077) |
|:---------------------------------|:-----------------------------------------------|:-------------------|:------------------ |:--------------------------------------------------|
| [Golang](binding/golang) | golang xdb 查询客户端 | :white_check_mark: | :white_check_mark: | [Lion](https://github.com/lionsoul2014) |
| [PHP](binding/php) | php xdb 查询客户端 | :white_check_mark: | :white_check_mark: | [Lion](https://github.com/lionsoul2014) |
| [Java](binding/java) | java xdb 查询客户端 | :white_check_mark: | :white_check_mark: | [Lion](https://github.com/lionsoul2014) |
| [C](binding/c) | POSIX C xdb 查询客户端 | :white_check_mark: | :white_check_mark: | [Lion](https://github.com/lionsoul2014) |
| [Lua_c](binding/lua_c) | lua c 扩展 xdb 查询客户端 | :white_check_mark: | :white_check_mark: | [Lion](https://github.com/lionsoul2014) |
| [Lua](binding/lua) | lua xdb 查询客户端 | :white_check_mark: | :x: | [Lion](https://github.com/lionsoul2014) |
| [Rust](binding/rust) | rust xdb 查询客户端 | :white_check_mark: | :white_check_mark: | [gongzhengyang](https://github.com/gongzhengyang) |
| [Python](binding/python) | python xdb 查询客户端 | :white_check_mark: | :x: | [厉害的花花](https://github.com/luckydog6132) |
| [Javascript](binding/javascript) | javascript (nodejs, typescript) xdb 查询客户端 | :white_check_mark: | :white_check_mark: | [Lion](https://github.com/lionsoul2014) |
| [Csharp](binding/csharp) | csharp xdb 查询客户端 | :white_check_mark: | :x: | [Alen Lee](https://github.com/malus2077) |
| [Erlang](binding/erlang) | erlang xdb 查询客户端 | :white_check_mark: | :x: | [malou](https://github.com/malou996) |
| [Nginx](binding/nginx) | nginx 扩展 xdb 查询客户端 | :white_check_mark: | :x: | [Wu Jian Ping](https://github.com/wujjpp) |
| [C++](binding/cpp) | C++ xdb 查询客户端 | :white_check_mark: | :x: | [Yunbin Liu](https://github.com/liuyunbin) |
以下工具链实现由社区开发者通过第三方仓库贡献：

154
binding/nodejs/ReadMe.md
View File

@ -1,155 +1,3 @@
# ip2region nodejs 查询客户端实现
## 使用方式
### 完全基于文件的查询
```javascript
// 导入包
const Searcher = require('.')
// 指定ip2region数据文件路径
const dbPath = 'ip2region.xdb file path'
try {
// 创建searcher对象
const searcher = Searcher.newWithFileOnly(dbPath)
// 查询
const data = await searcher.search('218.4.167.70')
// data: {region: '中国|0|江苏省|苏州市|电信', ioCount: 3, took: 1.342389}
} catch(e) {
console.log(e)
}
```
### 缓存 `VectorIndex` 索引
```javascript
// 导入包
const Searcher = require('.')
// 指定ip2region数据文件路径
const dbPath = 'ip2region.xdb file path'
try {
// 同步读取vectorIndex
const vectorIndex = Searcher.loadVectorIndexFromFile(dbPath)
// 创建searcher对象
const searcher = Searcher.newWithVectorIndex(dbPath, vectorIndex)
// 查询 await 或 promise均可
const data = await searcher.search('218.4.167.70')
// data: {region: '中国|0|江苏省|苏州市|电信', ioCount: 2, took: 0.402874}
} catch(e) {
console.log(e)
}
```
### 缓存整个 `xdb` 数据
```javascript
// 导入包
const Searcher = require('.')
// 指定ip2region数据文件路径
const dbPath = 'ip2region.xdb file path'
try {
// 同步读取buffer
const buffer = Searcher.loadContentFromFile(dbPath)
// 创建searcher对象
const searcher = Searcher.newWithBuffer(buffer)
// 查询 await 或 promise均可
const data = await searcher.search('218.4.167.70')
// data: {region:'中国|0|江苏省|苏州市|电信', ioCount: 0, took: 0.063833}
} catch(e) {
console.log(e)
}
```
## 查询测试
可以通过 `node ./tests/test.app.js` 命令来测试查询：
```shell
➜ nodejs git:(v2.0-for-nodejs) ✗ node ./tests/test.app.js --help
usage: Usage node test.app.js <agrs>
ip2region test app
optional arguments:
-h, --help show this help message and exit
-d DB, --db DB ip2region binary xdb file path, default: ../../data/ip2region.xdb
-c CACHE_POLICY, --cache-policy CACHE_POLICY
cache policy: file/vectorIndex/content, default: content
```
例如：使用默认的 data/ip2region.xdb 文件进行查询测试：
```shell
➜ nodejs git:(v2.0-for-nodejs) ✗ node ./tests/test.app.js
parameters:
dbPath: ../../data/ip2region.xdb
cache-policy: content
type 'quit' to exit
ip2region>> 1.2.3.4
{ region: '美国|0|华盛顿|0|谷歌', ioCount: 0, took: 54.606261 }
ip2region>>
```
输入 ip 即可进行查询测试，也可以分别设置 `cache-policy` 为 file/vectorIndex/content 来测试三种不同缓存实现的查询效果。
## bench 测试
```shell
➜ nodejs git:(v2.0-for-nodejs) ✗ node ./tests/bench.app.js --help
usage: Usage node test.app.js [command options]
ip2region benchmark app
optional arguments:
-h, --help show this help message and exit
--db DB ip2region binary xdb file path, default: ../../data/ip2region.xdb
--src SRC source ip text file path, default: ../../data/ip.merge.txt
--cache-policy CACHE_POLICY
cache policy: file/vectorIndex/content, default: content
```
例如：通过默认的 data/ip2region.xdb 和 data/ip.merge.txt 文件进行 bench 测试：
```shell
➜ nodejs git:(v2.0-for-nodejs) ✗ node ./tests/bench.app.js
options:
dbPath: ../../data/ip2region.xdb
src: ../../data/ip2region.xdb
cache-policy: content
Bench finished, {cachePolicy: content, total: 3417955, took: 20.591887765s, cost: 6.02462225658325μs/op}
```
可以通过分别设置 `cache-policy` 为 file/vectorIndex/content 来测试三种不同缓存实现的效果。
>Note: 注意 bench 使用的 src 文件要是生成对应 xdb 文件相同的源文件。
## 单元测试及覆盖率结果
```shell
➜ nodejs git:(v2.0-for-nodejs) ✗ npm run coverage
...
ip2region
✔ #newWithFileOnly and search
✔ #newWithVectorIndex and search
✔ #newWithBuffer and search
3 passing (6ms)
----------|---------|----------|---------|---------|----------------------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|----------------------------------
All files | 91.58 | 60.71 | 100 | 91.58 |
index.js | 91.58 | 60.71 | 100 | 91.58 | 61,75,90,146,152,187,193,207,215
----------|---------|----------|---------|---------|----------------------------------
```
Made with ♥ by Wu Jian Ping
请使用最新的 javascript binding：[javascript binding](../javascript/)

201
binding/nodejs/index.js
View File

@ -1,201 +0,0 @@
/*
* Created by Wu Jian Ping on - 2022/07/22.
*/
const fs = require('fs')
const VectorIndexSize = 8
const VectorIndexCols = 256
const VectorIndexLength = 256 * 256 * (4 + 4)
const SegmentIndexSize = 14
const IP_REGEX = /^((\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])\.){3}(\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])$/
const getStartEndPtr = Symbol('#getStartEndPtr')
const getBuffer = Symbol('#getBuffer')
const openFilePromise = Symbol('#openFilePromise')
class Searcher {
constructor (dbFile, vectorIndex, buffer) {
this._dbFile = dbFile
this._vectorIndex = vectorIndex
this._buffer = buffer
if (this._buffer) {
this._vectorIndex = this._buffer.subarray(256, 256 + VectorIndexLength)
}
}
async [getStartEndPtr] (idx, fd, ioStatus) {
if (this._vectorIndex) {
const sPtr = this._vectorIndex.readUInt32LE(idx)
const ePtr = this._vectorIndex.readUInt32LE(idx + 4)
return { sPtr, ePtr }
} else {
const buf = await this[getBuffer](256 + idx, 8, fd, ioStatus)
const sPtr = buf.readUInt32LE()
const ePtr = buf.readUInt32LE(4)
return { sPtr, ePtr }
}
}
async [getBuffer] (offset, length, fd, ioStatus) {
if (this._buffer) {
return this._buffer.subarray(offset, offset + length)
} else {
const buf = Buffer.alloc(length)
return new Promise((resolve, reject) => {
ioStatus.ioCount += 1
fs.read(fd, buf, 0, length, offset, (err) => {
if (err) {
reject(err)
} else {
resolve(buf)
}
})
})
}
}
[openFilePromise] (fileName) {
return new Promise((resolve, reject) => {
fs.open(fileName, 'r', (err, fd) => {
if (err) {
reject(err)
} else {
resolve(fd)
}
})
})
}
async search (ip) {
const startTime = process.hrtime()
const ioStatus = {
ioCount: 0
}
if (!isValidIp(ip)) {
throw new Error(`IP: ${ip} is invalid`)
}
let fd = null
if (!this._buffer) {
fd = await this[openFilePromise](this._dbFile)
}
const ps = ip.split('.')
const i0 = parseInt(ps[0])
const i1 = parseInt(ps[1])
const i2 = parseInt(ps[2])
const i3 = parseInt(ps[3])
const ipInt = i0 * 256 * 256 * 256 + i1 * 256 * 256 + i2 * 256 + i3
const idx = i0 * VectorIndexCols * VectorIndexSize + i1 * VectorIndexSize
const { sPtr, ePtr } = await this[getStartEndPtr](idx, fd, ioStatus)
let l = 0
let h = (ePtr - sPtr) / SegmentIndexSize
let result = null
while (l <= h) {
const m = (l + h) >> 1
const p = sPtr + m * SegmentIndexSize
const buff = await this[getBuffer](p, SegmentIndexSize, fd, ioStatus)
const sip = buff.readUInt32LE(0)
if (ipInt < sip) {
h = m - 1
} else {
const eip = buff.readUInt32LE(4)
if (ipInt > eip) {
l = m + 1
} else {
const dataLen = buff.readUInt16LE(8)
const dataPtr = buff.readUInt32LE(10)
const data = await this[getBuffer](dataPtr, dataLen, fd, ioStatus)
result = data.toString('utf-8')
break
}
}
}
if (fd) {
fs.close(fd,function(){})
}
const diff = process.hrtime(startTime)
const took = (diff[0] * 1e9 + diff[1]) / 1e3
return { region: result, ioCount: ioStatus.ioCount, took }
}
}
const _checkFile = dbPath => {
try {
fs.accessSync(dbPath, fs.constants.F_OK)
} catch (err) {
throw new Error(`${dbPath} ${err ? 'does not exist' : 'exists'}`)
}
try {
fs.accessSync(dbPath, fs.constants.R_OK)
} catch (err) {
throw new Error(`${dbPath} ${err ? 'is not readable' : 'is readable'}`)
}
}
const isValidIp = ip => {
return IP_REGEX.test(ip)
}
const newWithFileOnly = dbPath => {
_checkFile(dbPath)
return new Searcher(dbPath, null, null)
}
const newWithVectorIndex = (dbPath, vectorIndex) => {
_checkFile(dbPath)
if (!Buffer.isBuffer(vectorIndex)) {
throw new Error('vectorIndex is invalid')
}
return new Searcher(dbPath, vectorIndex, null)
}
const newWithBuffer = buffer => {
if (!Buffer.isBuffer(buffer)) {
throw new Error('buffer is invalid')
}
return new Searcher(null, null, buffer)
}
const loadVectorIndexFromFile = dbPath => {
const fd = fs.openSync(dbPath, 'r')
const buffer = Buffer.alloc(VectorIndexLength)
fs.readSync(fd, buffer, 0, VectorIndexLength, 256)
fs.close(fd,function(){})
return buffer
}
const loadContentFromFile = dbPath => {
const stats = fs.statSync(dbPath)
const buffer = Buffer.alloc(stats.size)
const fd = fs.openSync(dbPath, 'r')
fs.readSync(fd, buffer, 0, stats.size, 0)
fs.close(fd,function(){})
return buffer
}
module.exports = {
isValidIp,
loadVectorIndexFromFile,
loadContentFromFile,
newWithFileOnly,
newWithVectorIndex,
newWithBuffer
}

32
binding/nodejs/package.json
View File

@ -1,32 +0,0 @@
{
"name": "node-ip2region",
"version": "2.0.0",
"description": "official nodejs client of ip2region",
"main": "index.js",
"scripts": {
"lint": "eslint ./index.js",
"test": "mocha ./tests/function.test.js",
"coverage": "nyc npm run test",
"benchmark": "node ./tests/benchmark.js"
},
"author": "Wu Jian Ping",
"license": "ISC",
"devDependencies": {
"@types/chai": "^4.3.1",
"@types/node": "^18.0.6",
"argparse": "^2.0.1",
"benchmark": "^2.1.4",
"chai": "^4.3.6",
"eslint": "^8.20.0",
"eslint-config-standard": "^17.0.0",
"eslint-plugin-import": "^2.26.0",
"eslint-plugin-n": "^15.2.4",
"eslint-plugin-promise": "^6.0.0",
"linebyline": "^1.3.0",
"mocha": "^10.0.0",
"nyc": "^15.1.0"
},
"engines": {
"node": ">=8.0.0"
}
}

146
binding/nodejs/tests/bench.app.js
View File

@ -1,146 +0,0 @@
/*
* Created by Wu Jian Ping on - 2022/07/22.
*/
const Searcher = require('..')
const { ArgumentParser } = require('argparse')
const readline = require('linebyline')
// 处理输入参数
const parser = new ArgumentParser({
add_help: true,
description: 'ip2region benchmark app',
prog: 'node test.app.js',
usage: 'Usage %(prog)s [command options]'
})
parser.add_argument('--db', { help: 'ip2region binary xdb file path, default: ../../data/ip2region.xdb' })
parser.add_argument('--src', { help: 'source ip text file path, default: ../../data/ip.merge.txt' })
parser.add_argument('--cache-policy', { help: 'cache policy: file/vectorIndex/content, default: content' })
const args = parser.parse_args()
const dbPath = args.db || '../../data/ip2region.xdb'
const src = args.src || '../../data/ip.merge.txt'
const cachePolicy = args.cache_policy || 'content'
// 创建searcher对象
const createSearcher = () => {
let searcher = null
let vectorIndex = null
let buffer = null
switch (cachePolicy) {
case 'file':
searcher = Searcher.newWithFileOnly(dbPath)
break
case 'vectorIndex':
vectorIndex = Searcher.loadVectorIndexFromFile(dbPath)
searcher = Searcher.newWithVectorIndex(dbPath, vectorIndex)
break
default:
buffer = Searcher.loadContentFromFile(dbPath)
searcher = Searcher.newWithBuffer(buffer)
}
console.log('options: ')
console.log(` dbPath: ${dbPath}`)
console.log(` src: ${dbPath}`)
console.log(` cache-policy: ${cachePolicy}`)
console.log('')
return searcher
}
const ipToInt = ip => {
// 切割IP
const ps = ip.split('.')
// 将各段转成int
const i0 = parseInt(ps[0])
const i1 = parseInt(ps[1])
const i2 = parseInt(ps[2])
const i3 = parseInt(ps[3])
// 假如使用移位操作的话，这边可能产生负数
return i0 * 256 * 256 * 256 + i1 * 256 * 256 + i2 * 256 + i3
}
const intToIp = ip => {
const i0 = Math.floor(ip / (256 * 256 * 256))
const i1 = Math.floor((ip % (256 * 256 * 256)) / (256 * 256))
const i2 = Math.floor((ip % (256 * 256)) / 256)
const i3 = ip % 256
return `${i0}.${i1}.${i2}.${i3}`
}
const searcher = createSearcher()
// 开始时间
const startTime = process.hrtime()
let total = 0
// 程序主入口
const main = async () => {
const rl = readline(src)
rl
.on('line', async (line, lineCount, byteCount) => {
try {
const list = line.split('|')
const sip = list[0]
const eip = list[1]
if (!Searcher.isValidIp(sip)) {
throw new Error(`IP: ${sip} is invalid`)
}
if (!Searcher.isValidIp(eip)) {
throw new Error(`IP: ${eip} is invalid`)
}
const sipInt = ipToInt(sip)
const eipInt = ipToInt(eip)
if (sipInt > eipInt) {
throw new Error(`start ip(${sip}) should not be greater than end ip(${eip})`)
}
const mipInt = Math.floor((sipInt + eipInt) / 2)
const mip = intToIp(mipInt)
const mipLeftInt = Math.floor((sipInt + mipInt) / 2)
const mipLeft = intToIp(mipLeftInt)
const mipRightInt = Math.floor((mipInt + eipInt) / 2)
const mipRight = intToIp(mipRightInt)
const arr = [sip, mipLeft, mip, mipRight, eip]
for (let i = 0; i < arr.length; ++i) {
const target = arr[i]
const info = await searcher.search(target)
const region = list.slice(2, list.length).join('|')
// check the region info
if (region !== info.region) {
throw new Error(`failed search(${mip}) with (${region} != ${info.region})`)
}
total++
}
} catch (err) {
console.log(err)
process.exit(1)
}
})
.on('error', err => {
console.log(err)
process.exit(1)
})
}
process.on('exit', code => {
if (code === 0) {
// 这边只算个总时间就够了
const diff = process.hrtime(startTime)
const totalInNS = diff[0] * 1e9 + diff[1]
console.log(`Bench finished, {cachePolicy: ${cachePolicy}, total: ${total}, took: ${totalInNS / 1e9}s, cost: ${total === 0 ? 0 : (totalInNS / 1e3) / total}μs/op}`)
}
})
main()

38
binding/nodejs/tests/benchmark.js
View File

@ -1,38 +0,0 @@
/*
* Created by Wu Jian Ping on - 2022/07/22.
*/
const Benchmark = require('benchmark')
const path = require('path')
const Searcher = require('..')
const dbPath = path.join(__dirname, '..', '..', '..', 'data', 'ip2region.xdb')
const buffer = Searcher.loadContentFromFile(dbPath)
const searcher1 = Searcher.newWithBuffer(buffer)
const vectorIndex = Searcher.loadVectorIndexFromFile(dbPath)
const searcher2 = Searcher.newWithVectorIndex(dbPath, vectorIndex)
const searcher3 = Searcher.newWithFileOnly(dbPath)
const suite = new Benchmark.Suite()
suite
.add('#缓存整个xdb数据【搜索218.4.167.70】', async () => {
const ip = '218.4.167.70'
return searcher1.search(ip)
})
.add('#缓存VectorIndex索引【搜索218.4.167.70】', async () => {
const ip = '218.4.167.70'
return searcher2.search(ip)
})
.add('#完全基于文件的查询【搜索218.4.167.70】', async () => {
const ip = '218.4.167.70'
return searcher3.search(ip)
})
.on('cycle', function (event) {
console.log(String(event.target)) // eslint-disable-line
})
.on('complete', function () {
console.log('Fastest is ' + this.filter('fastest').map('name')) // eslint-disable-line
})
.run({ async: true })

33
binding/nodejs/tests/function.test.js
View File

@ -1,33 +0,0 @@
/*
* Created by Wu Jian Ping on - 2022/07/22.
*/
const { expect } = require('chai')
const path = require('path')
const Searcher = require('..')
const dbPath = path.join(__dirname, '..', '..', '..', 'data', 'ip2region.xdb')
const buffer = Searcher.loadContentFromFile(dbPath)
const searcher1 = Searcher.newWithBuffer(buffer)
const vectorIndex = Searcher.loadVectorIndexFromFile(dbPath)
const searcher2 = Searcher.newWithVectorIndex(dbPath, vectorIndex)
const searcher3 = Searcher.newWithFileOnly(dbPath)
describe('ip2region', () => {
it('#newWithFileOnly and search', async () => {
const d = await searcher3.search('218.4.167.70')
expect(d.region).equal('中国|0|江苏省|苏州市|电信')
})
it('#newWithVectorIndex and search', async () => {
const d = await searcher2.search('218.4.167.70')
expect(d.region).equal('中国|0|江苏省|苏州市|电信')
})
it('#newWithBuffer and search', async () => {
const d = await searcher1.search('218.4.167.70')
expect(d.region).equal('中国|0|江苏省|苏州市|电信')
})
})

79
binding/nodejs/tests/test.app.js
View File

@ -1,79 +0,0 @@
/*
* Created by Wu Jian Ping on - 2022/07/22.
*/
const Searcher = require('../')
const { ArgumentParser } = require('argparse')
// 处理输入参数
const parser = new ArgumentParser({
add_help: true,
description: 'ip2region test app',
prog: 'node test.app.js',
usage: 'Usage %(prog)s [command options]'
})
parser.add_argument('--db', { help: 'ip2region binary xdb file path, default: ../../data/ip2region.xdb' })
parser.add_argument('--cache-policy', { help: 'cache policy: file/vectorIndex/content, default: content' })
const args = parser.parse_args()
const dbPath = args.db || '../../data/ip2region.xdb'
const cachePolicy = args.cache_policy || 'content'
// 创建searcher对象
const createSearcher = () => {
let searcher = null
let vectorIndex = null
let buffer = null
switch (cachePolicy) {
case 'file':
searcher = Searcher.newWithFileOnly(dbPath)
break
case 'vectorIndex':
vectorIndex = Searcher.loadVectorIndexFromFile(dbPath)
searcher = Searcher.newWithVectorIndex(dbPath, vectorIndex)
break
default:
buffer = Searcher.loadContentFromFile(dbPath)
searcher = Searcher.newWithBuffer(buffer)
}
console.log('options: ')
console.log(` dbPath: ${dbPath}`)
console.log(` cache-policy: ${cachePolicy}`)
console.log('')
return searcher
}
// 从控制台读取用户一行输入
const readlineSync = () => {
return new Promise((resolve, reject) => {
process.stdin.resume()
process.stdin.on('data', data => {
process.stdin.pause()
resolve(data.toString('utf-8'))
})
})
}
const searcher = createSearcher()
const main = async () => {
console.log('type \'quit\' to exit')
while (true) {
process.stdout.write('ip2region>> ')
const ip = (await readlineSync()).trim()
if (ip === 'quit') {
process.exit(0)
} else {
try {
const response = await searcher.search(ip)
console.log(response)
} catch (err) {
console.log(err)
}
}
}
}
main()

45
binding/typescript/README.md
View File

@ -1,44 +1,3 @@
# ts-ip2region
# typescript xdb 查询客户端
TypeScript node client library for IP2Region
## Installation
Install the package with NPM
```bash
npm install ts-ip2region
# or
yarn add ts-ip2region
```
## Usage
```typescript
import { Searcher, CachePolicy, ISearcher } from 'ts-ip2region';
const dbPath = './ip2region.xdb';
const searcher: ISearcher = new Searcher(CachePolicy.Content, dbPath);
const ip = '8.8.8.8';
const regionInfo = await searcher.search(ip);
console.log(`IP: ${ip}, Region: ${regionInfo}`);
await searcher.close();
```
### Cache Policy Description
| Cache Policy | Description | Thread Safe |
|-------------------------|------------------------------------------------------------------------------------------------------------|-------------|
| CachePolicy.Content | Cache the entire `xdb` data. | Yes |
| CachePolicy.VectorIndex | Cache `vecotorIndex` to speed up queries and reduce system io pressure by reducing one fixed IO operation. | Yes |
| CachePolicy.File | Completely file-based queries | Yes |
### XDB File Description
Generate using [maker](https://github.com/lionsoul2014/ip2region/tree/master/maker/csharp), or [download](https://github.com/lionsoul2014/ip2region/blob/master/data/ip2region.xdb) pre-generated xdb files
## Contributing
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
Please make sure to update tests as appropriate.
## License
[Apache License 2.0](https://github.com/lionsoul2014/ip2region/blob/master/LICENSE.md)
请使用最新的 javascript binding：[javascript binding](../javascript/)

59
binding/typescript/package.json
View File

@ -1,59 +0,0 @@
{
"name": "ts-ip2region",
"version": "0.1.1",
"license": "Apache-2.0",
"author": "Alan",
"main": "dist/index.js",
"module": "dist/ip2region.esm.js",
"typings": "dist/index.d.ts",
"files": [
"dist",
"src"
],
"scripts": {
"analyze": "size-limit --why",
"build": "dts build",
"lint": "dts lint",
"prepare": "dts build",
"size": "size-limit",
"start": "dts watch",
"test": "dts test"
},
"husky": {
"hooks": {
"pre-commit": "dts lint"
}
},
"prettier": {
"printWidth": 80,
"semi": true,
"singleQuote": true,
"trailingComma": "es5"
},
"jest": {
"testEnvironment": "node"
},
"peerDependencies": {},
"engines": {
"node": ">=12"
},
"size-limit": [
{
"path": "dist/ip2region.cjs.production.min.js",
"limit": "10 KB"
},
{
"path": "dist/ip2region.esm.js",
"limit": "10 KB"
}
],
"devDependencies": {
"@size-limit/preset-small-lib": "^11.1.6",
"@tsconfig/recommended": "^1.0.8",
"dts-cli": "^2.0.5",
"husky": "^9.1.7",
"size-limit": "^11.1.6",
"tslib": "^2.8.1",
"typescript": "^5.7.2"
}
}

93
binding/typescript/src/cache/abstract-cache-strategy.ts vendored
View File

@ -1,93 +0,0 @@
import * as fs from 'fs/promises';
export abstract class AbstractCacheStrategy {
protected static readonly HEADER_INFO_LENGTH = 256;
protected static readonly VECTOR_INDEX_ROWS = 256;
protected static readonly VECTOR_INDEX_COLS = 256;
protected static readonly VECTOR_INDEX_SIZE = 8;
private fileHandle: fs.FileHandle | null = null;
private fileSize: number = 0;
private ioCounter = 0;
protected constructor(protected readonly xdbPath: string) {
// 构造不立即打开文件，懒加载
}
public get ioCount(): number {
return this.ioCounter;
}
protected incrementIoCount() {
this.ioCounter++;
}
/** 打开文件，若已打开则跳过 */
protected async openFile(): Promise<void> {
if (this.fileHandle === null) {
const fh = await fs.open(this.xdbPath, 'r');
this.fileHandle = fh;
const stat = await fh.stat();
this.fileSize = stat.size;
}
}
protected getFileSize(): number {
return this.fileSize;
}
/** 关闭文件 */
public async closeFile(): Promise<void> {
if (this.fileHandle) {
await this.fileHandle.close();
this.fileHandle = null;
}
}
/**
* 计算矢量索引的起始位置
* @param ip uint32
*/
protected getVectorIndexStartPos(ip: number): number {
const il0 = (ip >>> 24) & 0xFF;
const il1 = (ip >>> 16) & 0xFF;
const idx = il0 * AbstractCacheStrategy.VECTOR_INDEX_COLS * AbstractCacheStrategy.VECTOR_INDEX_SIZE
+ il1 * AbstractCacheStrategy.VECTOR_INDEX_SIZE;
return idx;
}
public abstract getVectorIndex(ip: number): Promise<Buffer>;
/**
* 异步读取文件数据（无需单资源锁，Node.js可安全并发读）
* @param offset 文件起始偏移
* @param length 读取字节数
*/
public async getData(offset: number, length: number): Promise<Buffer> {
await this.openFile();
if (!this.fileHandle) {
throw new Error('File handle is not available.');
}
// Node.js 通常会一次性读完，但仍保留循环以防偶发读不足
let totalBytesRead = 0;
const buffer = Buffer.allocUnsafe(length);
while (totalBytesRead < length) {
const { bytesRead } = await this.fileHandle.read({
buffer,
offset: totalBytesRead,
length: length - totalBytesRead,
position: offset + totalBytesRead
});
this.incrementIoCount();
if (bytesRead === 0) {
break; // 文件结束或出错
}
totalBytesRead += bytesRead;
}
return buffer.subarray(0, totalBytesRead);
}
}

22
binding/typescript/src/cache/cache-strategy-factory.ts vendored
View File

@ -1,22 +0,0 @@
import { CachePolicy } from "../models/cache-policy";
import { AbstractCacheStrategy } from "./abstract-cache-strategy";
import { ContentCacheStrategy } from "./content-cache-strategy";
import { FileCacheStrategy } from "./file-cache-strategy";
import { VectorIndexCacheStrategy } from "./vector-index-cache-strategy";
export class CacheStrategyFactory {
constructor(private readonly xdbPath: string) {}
public createCacheStrategy(cachePolicy: CachePolicy): AbstractCacheStrategy {
switch (cachePolicy) {
case CachePolicy.Content:
return new ContentCacheStrategy(this.xdbPath);
case CachePolicy.VectorIndex:
return new VectorIndexCacheStrategy(this.xdbPath);
case CachePolicy.File:
return new FileCacheStrategy(this.xdbPath);
default:
throw new Error(`Unknown cachePolicy: ${cachePolicy}`);
}
}
}

45
binding/typescript/src/cache/content-cache-strategy.ts vendored
View File

@ -1,45 +0,0 @@
import { AbstractCacheStrategy } from "./abstract-cache-strategy";
export class ContentCacheStrategy extends AbstractCacheStrategy {
private cacheData: Buffer | null = null;
constructor(xdbPath: string) {
super(xdbPath);
}
public async getVectorIndex(ip: number): Promise<Buffer> {
if (!this.cacheData) {
await this.ensureCacheLoaded();
}
if (!this.cacheData) {
throw new Error('Cache data is not loaded.');
}
const idx = this.getVectorIndexStartPos(ip);
return this.cacheData.subarray(
AbstractCacheStrategy.HEADER_INFO_LENGTH + idx,
AbstractCacheStrategy.HEADER_INFO_LENGTH + idx + AbstractCacheStrategy.VECTOR_INDEX_SIZE
);
}
public override async getData(offset: number, length: number): Promise<Buffer> {
if (!this.cacheData) {
await this.ensureCacheLoaded();
}
if (!this.cacheData) {
throw new Error('Cache data is not loaded.');
}
return this.cacheData.subarray(offset, offset + length);
}
private async ensureCacheLoaded(): Promise<void> {
if (this.cacheData !== null) {
return;
}
await this.openFile();
const fileSize = this.getFileSize();
this.cacheData = await super.getData(0, fileSize);
// 全量读取后立即关闭文件
await this.closeFile();
}
}

15
binding/typescript/src/cache/file-cache-strategy.ts vendored
View File

@ -1,15 +0,0 @@
import { AbstractCacheStrategy } from "./abstract-cache-strategy";
export class FileCacheStrategy extends AbstractCacheStrategy {
public constructor(xdbPath: string) {
super(xdbPath);
}
public async getVectorIndex(ip: number): Promise<Buffer> {
const idx = this.getVectorIndexStartPos(ip);
return this.getData(
AbstractCacheStrategy.HEADER_INFO_LENGTH + idx,
AbstractCacheStrategy.VECTOR_INDEX_SIZE
);
}
}

32
binding/typescript/src/cache/vector-index-cache-strategy.ts vendored
View File

@ -1,32 +0,0 @@
import { AbstractCacheStrategy } from "./abstract-cache-strategy";
export class VectorIndexCacheStrategy extends AbstractCacheStrategy {
private vectorIndex: Buffer | null = null;
constructor(xdbPath: string) {
super(xdbPath);
}
public async getVectorIndex(ip: number): Promise<Buffer> {
if (!this.vectorIndex) {
await this.ensureVectorIndexLoaded();
}
if (!this.vectorIndex) {
throw new Error('Vector index data not loaded.');
}
const idx = this.getVectorIndexStartPos(ip);
return this.vectorIndex.subarray(idx, idx + AbstractCacheStrategy.VECTOR_INDEX_SIZE);
}
private async ensureVectorIndexLoaded(): Promise<void> {
if (this.vectorIndex !== null) {
return;
}
await this.openFile();
const vectorLength = AbstractCacheStrategy.VECTOR_INDEX_ROWS
* AbstractCacheStrategy.VECTOR_INDEX_COLS
* AbstractCacheStrategy.VECTOR_INDEX_SIZE;
this.vectorIndex = await super.getData(AbstractCacheStrategy.HEADER_INFO_LENGTH, vectorLength);
// 保留 fileHandle 打开，因为后续可能还要继续读段数据
}
}

3
binding/typescript/src/index.ts
View File

@ -1,3 +0,0 @@
export * from './searcher/searcher';
export * from './searcher/ISearcher';
export * from './models/cache-policy';

5
binding/typescript/src/models/cache-policy.ts
View File

@ -1,5 +0,0 @@
export enum CachePolicy {
File = 'File',
VectorIndex = 'VectorIndex',
Content = 'Content'
}

6
binding/typescript/src/searcher/ISearcher.ts
View File

@ -1,6 +0,0 @@
export interface ISearcher {
search(ipStr: string): Promise<string | undefined>;
searchByUint32(ip: number): Promise<string | undefined>;
ioCount: number;
close(): Promise<void>;
}

84
binding/typescript/src/searcher/searcher.ts
View File

@ -1,84 +0,0 @@
import { AbstractCacheStrategy } from "../cache/abstract-cache-strategy";
import { CacheStrategyFactory } from "../cache/cache-strategy-factory";
import { CachePolicy } from "../models/cache-policy";
import { Util } from "../util/ip-util";
import { ISearcher } from "./ISearcher";
export class Searcher implements ISearcher {
private readonly cacheStrategy: AbstractCacheStrategy;
constructor(cachePolicy: CachePolicy, dbPath: string) {
const factory = new CacheStrategyFactory(dbPath);
this.cacheStrategy = factory.createCacheStrategy(cachePolicy);
}
public get ioCount(): number {
return this.cacheStrategy.ioCount;
}
/** 主动关闭 */
public async close(): Promise<void> {
await this.cacheStrategy.closeFile();
}
public async search(ipStr: string): Promise<string | undefined> {
const ip = Util.ipAddressToUInt32(ipStr);
return this.searchByUint32(ip);
}
/**
* 优化后的二分查找——一次性读 sPtr~ePtr 所有段数据，而不是循环 getData()
*/
public async searchByUint32(ip: number): Promise<string | undefined> {
const index = await this.cacheStrategy.getVectorIndex(ip);
if (index.length < 8) {
return undefined;
}
// 读取 sPtr, ePtr（uint32, 小端序）
const sPtr = index.readUInt32LE(0);
const ePtr = index.readUInt32LE(4);
const SEGMENT_INDEX_SIZE = 14;
const totalCount = Math.floor((ePtr - sPtr) / SEGMENT_INDEX_SIZE) + 1;
if (totalCount <= 0) {
return undefined;
}
// 一次性批量读取 [sPtr, ePtr] 范围内的所有 segment
const blockSize = totalCount * SEGMENT_INDEX_SIZE;
const blockBuff = await this.cacheStrategy.getData(sPtr, blockSize);
// 在内存中完成二分查找
let l = 0;
let h = totalCount - 1;
let dataLen = 0;
let dataPtr = 0;
while (l <= h) {
const mid = (l + h) >>> 1; // 中点
const offset = mid * SEGMENT_INDEX_SIZE;
const sip = blockBuff.readUInt32LE(offset);
const eip = blockBuff.readUInt32LE(offset + 4);
if (ip < sip) {
h = mid - 1;
} else if (ip > eip) {
l = mid + 1;
} else {
dataLen = blockBuff.readUInt16LE(offset + 8);
dataPtr = blockBuff.readUInt32LE(offset + 10);
break;
}
}
if (dataLen === 0) {
return undefined;
}
// 读取 region 数据
const regionBuff = await this.cacheStrategy.getData(dataPtr, dataLen);
return regionBuff.toString('utf8');
}
}

33
binding/typescript/src/util/ip-util.ts
View File

@ -1,33 +0,0 @@
export class Util {
public static isValidIpAddress(ipStr: string): boolean {
const octets = ipStr.split('.');
if (octets.length !== 4) {
return false;
}
for (const octet of octets) {
if (isNaN(Number(octet)) || Number(octet) < 0 || Number(octet) > 255) {
return false;
}
}
return true;
}
/**
* 把字符串形式的 IPv4 转为 uint32，小端序表示
* 例如 "1.2.3.4" -> 0x04030201
*/
public static ipAddressToUInt32(ipStr: string): number {
if (!Util.isValidIpAddress(ipStr)) {
throw new Error(`Invalid IPv4: ${ipStr}`);
}
const octets = ipStr.split('.').map(Number);
// 数据库是小端存储，如 1.2.3.4 => 0x04030201
return (
((octets[3] & 0xff) << 24) |
((octets[2] & 0xff) << 16) |
((octets[1] & 0xff) << 8) |
(octets[0] & 0xff)
) >>> 0;
}
}

31
binding/typescript/test/cache/content-cache-strategy.test.ts vendored
View File

@ -1,31 +0,0 @@
import { ContentCacheStrategy } from '../../src/cache/content-cache-strategy';
describe('ContentCacheStrategy Tests', () => {
let strategy: ContentCacheStrategy;
const dbPath = "../../data/ip2region.xdb";
beforeAll(() => {
strategy = new ContentCacheStrategy(dbPath);
});
afterAll(async () => {
await strategy.closeFile();
});
it('should load the entire file into memory', async () => {
// 首次访问会触发 ensureCacheLoaded
const vectorIndex = await strategy.getVectorIndex(0x01020304);
expect(vectorIndex.length).toBe(8); // VECTOR_INDEX_SIZE = 8
// 再次读取不会增加IO次数太多
const initIo = strategy.ioCount;
await strategy.getVectorIndex(0x05060708);
const newIo = strategy.ioCount;
expect(newIo).toBe(initIo); // 全量缓存后几乎不增加IO
});
it('should read arbitrary data after caching', async () => {
const data = await strategy.getData(0, 32); // 读取前32字节
expect(data.length).toBe(32);
});
});

36
binding/typescript/test/cache/file-cache-strategy.test.ts vendored
View File

@ -1,36 +0,0 @@
import { FileCacheStrategy } from '../../src/cache/file-cache-strategy';
describe('FileCacheStrategy Tests', () => {
let strategy: FileCacheStrategy;
//const dbPath = join(__dirname, '..', 'data', 'ip2region.xdb');
const dbPath = "../../data/ip2region.xdb";
beforeAll(() => {
strategy = new FileCacheStrategy(dbPath);
});
afterAll(async () => {
await strategy.closeFile();
});
it('should read vector index from file each time', async () => {
// 每次获取矢量索引，都会触发一次文件读取
const initIo = strategy.ioCount;
const vecIndex1 = await strategy.getVectorIndex(0x01020304);
expect(vecIndex1.length).toBe(8);
const afterIo1 = strategy.ioCount;
expect(afterIo1).toBeGreaterThan(initIo);
// 第二次读取仍增加IO，因为不缓存
const vecIndex2 = await strategy.getVectorIndex(0x05060708);
expect(vecIndex2.length).toBe(8);
const afterIo2 = strategy.ioCount;
expect(afterIo2).toBeGreaterThan(afterIo1);
});
it('should read arbitrary data directly from file', async () => {
const offset = 0;
const length = 64;
const data = await strategy.getData(offset, length);
expect(data.length).toBe(length);
});
});

40
binding/typescript/test/cache/vector-index-cache-strategy.test.ts vendored
View File

@ -1,40 +0,0 @@
import { VectorIndexCacheStrategy } from '../../src/cache/vector-index-cache-strategy';
describe('VectorIndexCacheStrategy Tests', () => {
let strategy: VectorIndexCacheStrategy;
//const dbPath = join(__dirname, '..', 'data', 'ip2region.xdb');
const dbPath = "../../data/ip2region.xdb";
beforeAll(() => {
strategy = new VectorIndexCacheStrategy(dbPath);
});
afterAll(async () => {
await strategy.closeFile();
});
it('should cache the vector index in memory', async () => {
// 首次调用会把矢量索引读到内存
const initIo = strategy.ioCount;
const vecIndex1 = await strategy.getVectorIndex(0x01020304);
expect(vecIndex1.length).toBe(8);
const afterIo1 = strategy.ioCount;
expect(afterIo1).toBeGreaterThan(initIo);
// 第二次获取矢量索引不会增加IO
const vecIndex2 = await strategy.getVectorIndex(0x05060708);
expect(vecIndex2.length).toBe(8);
const afterIo2 = strategy.ioCount;
expect(afterIo2).toBe(afterIo1);
});
it('should still read segment data from file as needed', async () => {
// 模拟读取部分数据段
const offset = 1024; // 示例偏移
const length = 32;
const segmentData = await strategy.getData(offset, length);
expect(segmentData.length).toBe(length);
// 这会增加IO
expect(strategy.ioCount).toBeGreaterThan(0);
});
});

39
binding/typescript/test/searcher/searcher.test.ts
View File

@ -1,39 +0,0 @@
import { Searcher } from '../../src/searcher/searcher';
import { CachePolicy } from '../../src/models/cache-policy';
describe('Searcher Tests', () => {
let searcher: Searcher;
beforeAll(() => {
// 指向一个真实存在的 xdb 文件（测试需要）
const dbPath = "../../data/ip2region.xdb";
// 测试用 ContentCacheStrategy（一次性载入文件）
searcher = new Searcher(CachePolicy.Content, dbPath);
});
afterAll(async () => {
await searcher.close();
});
it('should return valid data for a known IP', async () => {
const ipStr = '1.2.3.4';
const regionInfo = await searcher.search(ipStr);
expect(regionInfo).toBeDefined();
});
it('should return undefined for an invalid IP', async () => {
const invalidIp = '999.999.999.999';
await expect(searcher.search(invalidIp)).rejects.toThrow();
});
it('should track ioCount properly', async () => {
// ioCount 取决于缓存策略和查询次数
const initialCount = searcher.ioCount;
await searcher.search('8.8.8.8');
const afterCount = searcher.ioCount;
// ContentCacheStrategy：只读取一次，所以 ioCount 不变
expect(afterCount).toEqual(initialCount);
});
});

16
binding/typescript/test/util/ip-util.test.ts
View File

@ -1,16 +0,0 @@
import { Util } from '../../src/util/ip-util';
describe('IP Util Tests', () => {
it('should convert valid IPv4 string to uint32 correctly', () => {
const ipStr = '1.2.3.4';
const uint32 = Util.ipAddressToUInt32(ipStr);
// 1.2.3.4 小端序 => 0x04030201 => 67305985
expect(uint32).toBe(67305985);
});
it('should throw error for invalid IPv4 string', () => {
expect(() => {
Util.ipAddressToUInt32('256.256.256.256');
}).toThrow();
});
});

8
binding/typescript/tsconfig.json
View File

@ -1,8 +0,0 @@
{
"extends": "@tsconfig/recommended/tsconfig.json",
"include": ["src", "types"],
"compilerOptions": {
"module": "esnext",
"allowImportingTsExtensions": false
}
}

6370
binding/typescript/yarn.lock
View File

File diff suppressed because it is too large Load Diff