Skip to content

Commit 4498915

Browse files
masklinnmasklinn
committed
Atopt ua-parser-rs from uap-rust
Originally, building ua-parser-rs alongside uap-rust seemed to make sense, but over time I've soured on it: although it's largely Rust code: - The interface for python and (should) follow Python API design principles. - It's bound much more to ua-parser/uap-python (as an implementation detail thereof) than to ua-parser/uap-rust (which is just a dependency). - The tests are pure Python. - The CI and release pipeline are completely pipeline coded (testing and releasing for various python runtimes). And in case the Python and Rust teams split in the future, it probably doesn't make sense for the python team to have to go through the rust team to tune the extension, even if they may want / need counsel from that team. Adoption notes: - Publishing via maturin dropped, as it's deprecated. - Rust checks added to tox. - Dropped explicitly building and installing the ua-parser-rs as that's what `pip install ./ua-parser-rs` should be doing. - Because the wheels workflow is tricky, it can be run on a PR by tagging it allow running the wheels workflow in a PR. It's not enabled by default (or even when touching `ua-parser-rs`) as the cost of that job is enormous. - During wheels build, *only* the `-regex` tests are run, with `pytest-error-for-skips` to make sure the tests are not skipped (aka `ua-parser-rs` is properly installed).
1 parent 52b7c6c commit 4498915

File tree

13 files changed

+821
-6
lines changed

13 files changed

+821
-6
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ name: CI
22

33
on:
44
push:
5+
branches:
56
pull_request:
67

78
permissions: {}
@@ -128,7 +129,7 @@ jobs:
128129
- run: python -mpip install pytest pyyaml
129130
- run: python -mpip install ./ua-parser-builtins
130131
# install rs accelerator if available, ignore if not
131-
- run: python -mpip install ua-parser-rs || true
132+
- run: python -mpip install ./ua-parser-rs || true
132133
# re2 is basically impossible to install from source so don't
133134
# bother, and suppress installation failure so the test does
134135
# not fail (re2 tests will just be skipped for versions /

‎.github/workflows/release-wheels.yml‎

Lines changed: 256 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,256 @@
1+
name: Wheels
2+
3+
on:
4+
pull_request:
5+
types: [opened, reopened, labeled, synchronize]
6+
workflow_dispatch:
7+
inputs:
8+
release:
9+
description: 'Push wheels to pypi'
10+
type: boolean
11+
default: false
12+
required: true
13+
14+
permissions: {}
15+
16+
concurrency:
17+
group: ${{ github.workflow }}-${{ github.ref }}
18+
cancel-in-progress: true
19+
20+
jobs:
21+
py-wheels-matrix:
22+
name: "generate build matrix"
23+
runs-on: ubuntu-latest
24+
if: >
25+
github.event_name == 'workflow_dispatch' ||
26+
(
27+
github.event_name == 'pull_request' &&
28+
github.event.type == 'labeled' &&
29+
github.event.label.name == 'check-wheels'
30+
) ||
31+
(
32+
github.event_name == 'pull_request' &&
33+
github.event.type != 'labeled' &&
34+
contains(github.event.pull_request.labels.*.name, 'check-wheels')
35+
)
36+
outputs:
37+
matrix: ${{ steps.make-matrix.outputs.matrix }}
38+
steps:
39+
- id: make-matrix
40+
shell: python
41+
name: generate matrix
42+
run: |
43+
import itertools
44+
import json
45+
import os
46+
import pprint
47+
48+
builder = {
49+
('linux', 'x86_64'): 'ubuntu-latest',
50+
('linux', 'aarch64'): 'ubuntu-24.04-arm',
51+
('musllinux', 'x86_64'): 'ubuntu-latest',
52+
('musllinux', 'aarch64'): 'ubuntu-24.04-arm',
53+
('macos', 'x86_64'): 'macos-15-intel',
54+
('macos', 'aarch64'): 'macos-latest',
55+
('windows', 'x86_64'): 'windows-latest',
56+
('windows', 'aarch64'): 'windows-11-arm',
57+
}
58+
59+
matrix = [
60+
d
61+
for d in map(dict, itertools.product(
62+
(('python-version', v) for v in ["3.x", "3.14t", "pypy-3.11", "graalpy-25"]),
63+
(('arch', a) for a in ["x86_64", "aarch64"]),
64+
(('platform', p) for p in ["linux", "musllinux", "windows", "macos"])
65+
))
66+
# on windows, only cpython has arm builds (?)
67+
if d['python-version'].startswith('3.') \
68+
or d['platform'] != 'windows' \
69+
or d['arch'] != 'aarch64'
70+
]
71+
for job in matrix:
72+
match job['platform']:
73+
case 'linux':
74+
job['manylinux'] = 'auto'
75+
job['args'] = ' --zig'
76+
case 'mussllinux':
77+
job['manylinux'] = 'musllinux_1_2'
78+
79+
job['runs'] = builder[job['platform'], job['arch']]
80+
81+
with open(os.environ['GITHUB_OUTPUT'], 'w') as f:
82+
f.write("matrix=")
83+
json.dump({'include': matrix}, f)
84+
f.flush()
85+
86+
py-release-wheels:
87+
needs: [py-wheels-matrix]
88+
strategy:
89+
matrix: ${{fromJson(needs.py-wheels-matrix.outputs.matrix)}}
90+
91+
runs-on: ${{ matrix.runs }}
92+
93+
steps:
94+
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # 6.0.1
95+
with:
96+
persist-credentials: false
97+
- uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # 6.1.0
98+
with:
99+
python-version: ${{ matrix.python-version }}
100+
# windows/arm doesn't have a rust toolchain by default
101+
- if: matrix.platform == 'windows' && matrix.arch == 'aarch64'
102+
uses: actions-rust-lang/setup-rust-toolchain@9d7e65c320fdb52dcd45ffaa68deb6c02c8754d9 # 1.12.0
103+
- name: Build wheels
104+
uses: PyO3/maturin-action@04ac600d27cdf7a9a280dadf7147097c42b757ad # 1.50.1
105+
with:
106+
args: --release --out dist -m ua-parser-rs/Cargo.toml -i python ${{ matrix.args }}
107+
sccache: 'true'
108+
manylinux: ${{ matrix.manylinux }}
109+
- name: Upload wheels
110+
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # 7.0.0
111+
with:
112+
name: ua-parser-rs-${{ matrix.platform }}-${{ matrix.arch }}-${{ matrix.python-version }}
113+
path: dist/*
114+
retention-days: 1
115+
compression-level: 0
116+
117+
py-release-sdist:
118+
runs-on: ubuntu-latest
119+
steps:
120+
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # 6.0.1
121+
with:
122+
persist-credentials: false
123+
- name: Build sdist
124+
uses: PyO3/maturin-action@04ac600d27cdf7a9a280dadf7147097c42b757ad # 1.50.1
125+
with:
126+
command: sdist
127+
args: --out dist -m ua-parser-rs/Cargo.toml
128+
- name: Upload sdist
129+
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # 7.0.0
130+
with:
131+
name: wheels-sdist
132+
path: dist
133+
134+
py-release-tests:
135+
needs: py-release-wheels
136+
137+
strategy:
138+
matrix:
139+
python-version:
140+
- "3.10"
141+
- "3.11"
142+
- "3.12"
143+
- "3.13"
144+
- "3.14"
145+
- "3.14t"
146+
- "pypy-3.11"
147+
- "graalpy-25"
148+
platform:
149+
- linux
150+
# probably requires a custom image of some sort
151+
# - musllinux
152+
- windows
153+
- macos
154+
arch:
155+
- x86_64
156+
- aarch64
157+
158+
exclude:
159+
- platform: windows
160+
python-version: 3.10
161+
arch: aarch64
162+
- platform: windows
163+
arch: aarch64
164+
python-version: pypy-3.11
165+
- platform: windows
166+
python-version: graalpy-25
167+
168+
include:
169+
- wheel: "3.x"
170+
- python-version: "3.14t"
171+
wheel: "3.14t"
172+
- python-version: "pypy-3.11"
173+
wheel: "pypy-3.11"
174+
- python-version: "graalpy-25"
175+
wheel: "graalpy-25"
176+
177+
- runner: ubuntu-latest
178+
- arch: aarch64
179+
runner: ubuntu-24.04-arm
180+
- platform: windows
181+
runner: windows-latest
182+
- platform: windows
183+
arch: aarch64
184+
runner: windows-11-arm
185+
- platform: macos
186+
runner: macos-latest
187+
- platform: macos
188+
arch: x86_64
189+
runner: macos-15-intel
190+
- opts: ""
191+
- python-version: graalpy-25
192+
opts: "--experimental-options --engine.CompileOnly='~tregex re'"
193+
194+
runs-on: ${{ matrix.runner }}
195+
196+
steps:
197+
- name: Checkout working copy
198+
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # 6.0.1
199+
with:
200+
submodules: true
201+
persist-credentials: false
202+
- name: Set up Python ${{ matrix.python-version }}
203+
uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # 6.1.0
204+
with:
205+
python-version: ${{ matrix.python-version }}
206+
allow-prereleases: true
207+
- name: Retrieve wheel
208+
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # 8.0.1
209+
with:
210+
name: ua-parser-rs-${{ matrix.platform }}-${{ matrix.arch }}-${{ matrix.wheel }}
211+
path: dist
212+
- name: Update pip
213+
run: python -mpip install --upgrade pip
214+
- name: Maybe install libyaml-dev
215+
if: startsWith(matrix.runs, 'ubuntu-latest')
216+
run: |
217+
# if binary wheels are not available for the current
218+
# package install libyaml-dev so we can install pyyaml
219+
# from source
220+
if ! pip download --only-binary :all: pyyaml > /dev/null 2>&1; then
221+
sudo apt install libyaml-dev
222+
fi
223+
- name: Install test dependencies
224+
run: python -mpip install pytest pyyaml pytest-error-for-skips
225+
- name: Install wheel
226+
run: python -mpip install --only-binary ':all:' --no-index --find-links dist ua_parser_rs
227+
- name: Install package
228+
run: python -mpip install .
229+
- name: Run tests
230+
run: python ${{ matrix.opts }} -m pytest -v -Werror -Wignore::ImportWarning --doctest-glob="*.rst" -ra --error-for-skips -k '(-regex)'
231+
232+
py-release:
233+
name: Release
234+
runs-on: ubuntu-latest
235+
needs: [py-release-tests, py-release-sdist]
236+
if: github.event.name == 'workflow_dispatch'
237+
permissions:
238+
id-token: write
239+
environment: release
240+
steps:
241+
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # 8.0.1
242+
with:
243+
path: dist/
244+
merge-multiple: true # dump every wheel file in the same directory
245+
- name: Publish to PyPI
246+
if: (inputs.release)
247+
uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # 1.13.0
248+
with:
249+
verbose: true
250+
- name: Publish to TestPyPI
251+
if: (!inputs.release)
252+
uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # 1.13.0
253+
with:
254+
repository-url: https://test.pypi.org/legacy/
255+
skip-existing: true
256+
verbose: true

‎.github/workflows/rs-checks.yml‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
name: rs checks
2+
3+
on:
4+
pull_request:
5+
push:
6+
branches:
7+
8+
permissions: {}
9+
10+
jobs:
11+
rs-checks:
12+
runs-on: ubuntu-latest
13+
steps:
14+
- name: Checkout working copy
15+
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # 6.0.1
16+
with:
17+
submodules: true
18+
fetch-depth: 0
19+
persist-credentials: false
20+
- name: cargo check
21+
run: cargo check --manifest-path ua-parser-rs/Cargo.toml
22+
- name: cargo clippy # don't run clippy if check fails
23+
run: cargo clippy --manifest-path ua-parser-rs/Cargo.toml
24+
- name: cargo fmt
25+
if: always()
26+
run: cargo fmt --check --manifest-path ua-parser-rs/Cargo.toml

‎.github/workflows/zizmor.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ name: Zizmor
22

33
on:
44
push:
5+
branches:
56
pull_request:
67

78
permissions: {}

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,6 @@ tmp/
99
regexes.yaml
1010
_regexes.py
1111
doc/_build
12+
uv.lock
13+
ua-parser-rs/Cargo.lock
14+
ua-parser-rs/target

‎tests/test_core.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,8 @@
8080
CORE_DIR / "tests" / "test_ua.yaml",
8181
CORE_DIR / "test_resources" / "firefox_user_agent_strings.yaml",
8282
CORE_DIR / "test_resources" / "pgts_browser_list.yaml",
83+
CORE_DIR / "test_resources" / "opera_mini_user_agent_strings.yaml",
84+
CORE_DIR / "test_resources" / "podcasting_user_agent_strings.yaml",
8385
],
8486
ids=attrgetter("stem"),
8587
)

‎tox.ini‎

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,10 +13,11 @@ package = wheel
1313
wheel_build_env = .pkg
1414
# for extra deps
1515
# extras =
16+
allowlist_externals = cargo
1617
deps =
1718
pytest
1819
pyyaml
19-
ua-parser-rs
20+
./ua-parser-rs
2021
./ua-parser-builtins
2122
commands =
2223
pytest -Werror --doctest-glob="*.rst" {posargs}
@@ -27,30 +28,37 @@ deps =
2728
pytest
2829
pyyaml
2930
google-re2
30-
ua-parser-rs
31+
./ua-parser-rs
3132
./ua-parser-builtins
3233

3334
[testenv:check]
3435
labels = check
3536
package = skip
3637
deps = ruff
37-
commands = ruff check {posargs}
38+
commands =
39+
cargo clippy --manifest-path ua-parser-rs/Cargo.toml {posargs}
40+
ruff check {posargs}
3841

3942
[testenv:format]
4043
description = Runs the formatter (just showing errors by default)
4144
labels = check
4245
package = skip
4346
deps = ruff
44-
commands = ruff format {posargs:--diff}
47+
commands =
48+
cargo fmt --manifest-path ua-parser-rs/Cargo.toml {posargs:--check}
49+
ruff format {posargs:--diff}
4550

4651
[testenv:typecheck]
4752
labels = check
4853
package = skip
4954
deps =
5055
mypy
5156
types-PyYaml
57+
./ua-parser-rs
5258
./ua-parser-builtins
53-
commands = mypy {posargs}
59+
commands =
60+
cargo check --manifest-path ua-parser-rs/Cargo.toml
61+
mypy {posargs}
5462

5563
[testenv:docs]
5664
description = Builds the documentation

‎ua-parser-rs/Cargo.toml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
[package]
2+
name = "ua-parser-rs"
3+
version = "0.1.4"
4+
edition = "2024"
5+
license = "Apache 2.0"
6+
description = "A native accelerator for uap-python"
7+
repository = "https://github.com/ua-parser/uap-rust/"
8+
homepage = "https://github.com/ua-parser/uap-rust/blob/main/ua-parser/"
9+
authors = ["masklinn <uap@masklinn.net>"]
10+
11+
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
12+
[lib]
13+
name = "ua_parser_rs"
14+
crate-type = ["cdylib"]
15+
16+
[dependencies]
17+
pyo3 = { version = "0.27", features = ["extension-module", "abi3", "abi3-py310"] }
18+
ua-parser = "0.2.2"

0 commit comments

Comments
 (0)