Compare commits

..

12 Commits

Author SHA1 Message Date
Adeeb Shihadeh
ec99b49198 rename openssl package to openssl3 (#17) 2026-02-24 11:20:46 -08:00
Adeeb Shihadeh
31b96deac5 add openssl package (#16)
Vendors OpenSSL 3.4.1 as static libraries (libcrypto.a, libssl.a) for
openpilot, following the same pattern as other vendored packages.

Uses module name openssl3 to avoid case-insensitive filesystem conflict
with pyOpenSSL's OpenSSL module on macOS.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 10:53:01 -08:00
Adeeb Shihadeh
6fe17e7b31 add libjpeg-turbo package (#15)
Vendors libjpeg-turbo 3.1.0 as a static library for openpilot, following
the same pattern as zstd/ncurses/zeromq.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 10:09:19 -08:00
Adeeb Shihadeh
2377af3f1f fix git-lfs on macOS 2026-02-23 22:59:08 -08:00
Adeeb Shihadeh
b9f89a5a5d disable LTO for zstd to fix cross-version linking on macOS
LTO bitcode is compiler-version-specific. The wheel built with
clang 17 produces bitcode that clang 16 (used in openpilot CI)
cannot read. Disable IPO to produce standard object files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 22:46:55 -08:00
Adeeb Shihadeh
fd4ae72793 add zstd and ncurses packages (#14) 2026-02-23 22:34:46 -08:00
Adeeb Shihadeh
dfa87aa31a add python3-dev package (#13)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 21:46:40 -08:00
Adeeb Shihadeh
71d4d1d3a1 downgrade cppcheck to 2.16.0 (#12) 2026-02-23 11:47:16 -08:00
Adeeb Shihadeh
f117a0e60d add cppcheck package (#10)
Cppcheck 2.19.1 static analysis tool, built from source. Includes
the MISRA C addon, cfg files, and platform definitions for opendbc
and panda safety analysis.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 10:18:33 -08:00
Adeeb Shihadeh
bf379a6268 add eigen package (#9)
Header-only Eigen 3.4.0 C++ linear algebra library, downloaded from
GitLab release. Exposes INCLUDE_DIR for scons integration.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 21:39:17 -08:00
Adeeb Shihadeh
fcd4d49232 better README (#5) 2026-02-22 21:34:30 -08:00
Adeeb Shihadeh
9a30c4f717 update .gitignore 2026-02-22 21:29:04 -08:00
31 changed files with 1038 additions and 13 deletions

8
.gitignore vendored
View File

@@ -1,5 +1,7 @@
.git/ .git/
repo/ repo/
.venv/
__pycache__/
# agents # agents
.claude/ .claude/
@@ -9,16 +11,12 @@ TASK.md
# build artifacts (applies to all packages) # build artifacts (applies to all packages)
build/ build/
dist/ dist/
install/
*.egg-info/ *.egg-info/
# downloaded toolchain binaries # downloaded toolchain binaries
gcc-arm-none-eabi/gcc_arm_none_eabi/toolchain/ gcc-arm-none-eabi/gcc_arm_none_eabi/toolchain/
# built capnproto
capnproto/capnproto/install/
# built ffmpeg
ffmpeg/ffmpeg/install/
# downloaded git-lfs # downloaded git-lfs
git-lfs/git_lfs/bin/ git-lfs/git_lfs/bin/

View File

@@ -1,14 +1,24 @@
# dependencies # dependencies
a central repo for managing and vendoring third party dependencies for all comma projects. a central repo for managing and [vendoring](https://htmx.org/essays/vendoring/) third party dependencies for all comma projects.
all of our projects are python projects, so each of these gets packaged as a pip project. since all our projects are Python, we wrap each vendored dependency as a pip package. `git clone` and `uv sync` is all you need.
this vendoring serves a few goals: motivations for this approach
- all dependencies are centrally managed here - `apt-get` is slow
- we can slim them down to only what we need - `apt-get` updates its packages on a schedule we don't control
- all platforms get the same versions installed - `apt-get` package versions don't match `brew` versions
- tighter control of distribution for fast installs (e.g. Ubuntu's `apt-get` is slow) - `apt-get` doesn't come with Arch Linux
- `apt-get` doesn't always have the exact package we need
- `apt-get` packages are often bloated
<!--
this critically adds friction to adding dependencies to our project
- `apt-get` installing a package is easy. is it 1MB, 10MB, or 100MB? no idea.
- `apt-get` installing a package is much easier than adding it here. how much do you want it?
-->
`uv`, as opposed to `apt-get`, `brew`, and friends, is fast and already used in our projects.
we target the following platforms: we target the following platforms:
- Linux x86_64 - Linux x86_64
@@ -16,3 +26,30 @@ we target the following platforms:
- Darwin aarch64 (Apple Silicon) - Darwin aarch64 (Apple Silicon)
contributions welcome for other platforms! contributions welcome for other platforms!
## packages
| package | description |
|-------------------|--------------------------------------------------------------------------|
| gcc-arm-none-eabi | builds [panda](https://github.com/commaai/panda) firmware for STM32 MCUs |
| capnproto | message serialization for openpilot |
| ffmpeg | video encode and decode for openpilot |
| git-lfs | for tracking large files in openpilot |
| zeromq | bridging the openpilot IPC between different hosts |
## usage
```python
dependencies = [
"capnproto @ git+https://github.com/commaai/dependencies.git@releases#subdirectory=capnproto",
"ffmpeg @ git+https://github.com/commaai/dependencies.git@releases#subdirectory=ffmpeg",
]
```
## workflow
to add a new package:
* start a new top-level directory as a new package
* `./test.sh` tests the building of all packages
* on pushes to `master`, wheels are built for our target platforms and pushed to a GitHub release
* the `releases` branch contains shim packages that allow pointing to a git branch and always getting the appropriate wheel for your platform

42
cppcheck/build.sh Executable file
View File

@@ -0,0 +1,42 @@
#!/usr/bin/env bash
set -e
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
cd "$DIR"
VERSION="2.16.0"
INSTALL_DIR="$DIR/cppcheck/install"
# Idempotent: skip if already built
if [ -x "$INSTALL_DIR/cppcheck" ]; then
echo "cppcheck already present, skipping build."
exit 0
fi
NJOBS="$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 2)"
# Clone
if [ ! -d "cppcheck-src" ]; then
git clone --depth 1 --branch "$VERSION" https://github.com/danmar/cppcheck.git cppcheck-src
fi
# Build
cd cppcheck-src
make MATCHCOMPILER=yes CXXFLAGS="-O2" -j"$NJOBS"
cd "$DIR"
# Install
rm -rf "$INSTALL_DIR"
mkdir -p "$INSTALL_DIR"
cp cppcheck-src/cppcheck "$INSTALL_DIR/"
cp -r cppcheck-src/addons "$INSTALL_DIR/"
cp -r cppcheck-src/cfg "$INSTALL_DIR/"
cp -r cppcheck-src/platforms "$INSTALL_DIR/"
strip "$INSTALL_DIR/cppcheck" 2>/dev/null || true
# Clean up
rm -rf cppcheck-src
echo "Installed cppcheck to $INSTALL_DIR"
du -sh "$INSTALL_DIR"

View File

@@ -0,0 +1,16 @@
import os
import sys
DIR = os.path.join(os.path.dirname(__file__), "install")
def _run():
binary = os.path.join(DIR, "cppcheck")
os.execvp(binary, ["cppcheck"] + sys.argv[1:])
def smoketest():
import subprocess
binary = os.path.join(DIR, "cppcheck")
result = subprocess.run([binary, "--version"], capture_output=True, text=True, check=True)
print(result.stdout.strip())

18
cppcheck/pyproject.toml Normal file
View File

@@ -0,0 +1,18 @@
[build-system]
requires = ["setuptools>=64", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "cppcheck"
version = "2.16.0"
description = "Cppcheck static analysis tool"
requires-python = ">=3.8"
[project.scripts]
cppcheck = "cppcheck:_run"
[tool.setuptools.packages.find]
include = ["cppcheck*"]
[tool.setuptools.package-data]
cppcheck = ["install/**/*"]

61
cppcheck/setup.py Normal file
View File

@@ -0,0 +1,61 @@
import os
import platform
import subprocess
from setuptools.command.build_py import build_py
try:
from wheel.bdist_wheel import bdist_wheel
except ImportError:
bdist_wheel = None
class BuildCppcheck(build_py):
"""Run build.sh to compile cppcheck before collecting package data."""
def run(self):
pkg_dir = os.path.dirname(os.path.abspath(__file__))
marker = os.path.join(pkg_dir, "cppcheck", "install", "cppcheck")
if not os.path.exists(marker):
build_script = os.path.join(pkg_dir, "build.sh")
subprocess.check_call(["bash", build_script], cwd=pkg_dir)
super().run()
cmdclass = {"build_py": BuildCppcheck}
if bdist_wheel is not None:
class PlatformWheel(bdist_wheel):
"""Produce a platform-specific, Python-version-agnostic wheel."""
def finalize_options(self):
super().finalize_options()
self.root_is_pure = False
def get_tag(self):
system = platform.system()
machine = platform.machine()
if system == "Linux":
plat = f"linux_{machine}"
elif system == "Darwin":
plat = "macosx_11_0_arm64"
else:
plat = f"{system.lower()}_{machine}"
return "py3", "none", plat
cmdclass["bdist_wheel"] = PlatformWheel
def setup():
from setuptools import setup as _setup
_setup(cmdclass=cmdclass)
if __name__ == "__main__":
setup()

34
eigen/build.sh Executable file
View File

@@ -0,0 +1,34 @@
#!/usr/bin/env bash
set -e
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
cd "$DIR"
VERSION="3.4.0"
INSTALL_DIR="$DIR/eigen/install"
# Idempotent: skip if already present
if [ -d "$INSTALL_DIR/eigen3/Eigen" ]; then
echo "eigen already present, skipping download."
exit 0
fi
TARBALL="eigen-${VERSION}.tar.gz"
URL="https://gitlab.com/libeigen/eigen/-/archive/${VERSION}/${TARBALL}"
echo "Downloading Eigen ${VERSION} ..."
curl -fSL -o "$TARBALL" "$URL"
echo "Extracting headers ..."
rm -rf "$INSTALL_DIR"
mkdir -p "$INSTALL_DIR/eigen3"
# Extract only the Eigen/ and unsupported/Eigen/ header directories
tar --strip-components=1 -xzf "$TARBALL" -C "$INSTALL_DIR/eigen3" \
"eigen-${VERSION}/Eigen" \
"eigen-${VERSION}/unsupported"
rm -f "$TARBALL"
echo "Installed eigen to $INSTALL_DIR"
du -sh "$INSTALL_DIR"

9
eigen/eigen/__init__.py Normal file
View File

@@ -0,0 +1,9 @@
import os
DIR = os.path.join(os.path.dirname(__file__), "install")
INCLUDE_DIR = DIR
LIB_DIR = DIR # header-only; no libraries
def smoketest():
assert os.path.isdir(os.path.join(DIR, "eigen3", "Eigen")), "Eigen headers not found"

15
eigen/pyproject.toml Normal file
View File

@@ -0,0 +1,15 @@
[build-system]
requires = ["setuptools>=64", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "eigen"
version = "3.4.0"
description = "Eigen C++ linear algebra headers"
requires-python = ">=3.8"
[tool.setuptools.packages.find]
include = ["eigen*"]
[tool.setuptools.package-data]
eigen = ["install/**/*"]

61
eigen/setup.py Normal file
View File

@@ -0,0 +1,61 @@
import os
import platform
import subprocess
from setuptools.command.build_py import build_py
try:
from wheel.bdist_wheel import bdist_wheel
except ImportError:
bdist_wheel = None
class BuildEigen(build_py):
"""Run build.sh to download Eigen headers before collecting package data."""
def run(self):
pkg_dir = os.path.dirname(os.path.abspath(__file__))
marker = os.path.join(pkg_dir, "eigen", "install", "eigen3", "Eigen")
if not os.path.isdir(marker):
build_script = os.path.join(pkg_dir, "build.sh")
subprocess.check_call(["bash", build_script], cwd=pkg_dir)
super().run()
cmdclass = {"build_py": BuildEigen}
if bdist_wheel is not None:
class PlatformWheel(bdist_wheel):
"""Produce a platform-specific, Python-version-agnostic wheel."""
def finalize_options(self):
super().finalize_options()
self.root_is_pure = False
def get_tag(self):
system = platform.system()
machine = platform.machine()
if system == "Linux":
plat = f"linux_{machine}"
elif system == "Darwin":
plat = "macosx_11_0_arm64"
else:
plat = f"{system.lower()}_{machine}"
return "py3", "none", plat
cmdclass["bdist_wheel"] = PlatformWheel
def setup():
from setuptools import setup as _setup
_setup(cmdclass=cmdclass)
if __name__ == "__main__":
setup()

View File

@@ -49,7 +49,6 @@ else
fi fi
chmod +x "$INSTALL_DIR/git-lfs" chmod +x "$INSTALL_DIR/git-lfs"
strip "$INSTALL_DIR/git-lfs" 2>/dev/null || true
rm -f "$FILENAME" rm -f "$FILENAME"

56
libjpeg/build.sh Executable file
View File

@@ -0,0 +1,56 @@
#!/usr/bin/env bash
set -e
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
cd "$DIR"
VERSION="3.1.0"
INSTALL_DIR="$DIR/libjpeg/install"
# Idempotent: skip if already built
if [ -f "$INSTALL_DIR/lib/libjpeg.a" ]; then
echo "libjpeg already present, skipping build."
exit 0
fi
NJOBS="$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 2)"
# Clone
if [ ! -d "libjpeg-turbo-src" ]; then
git clone --depth 1 --branch "${VERSION}" https://github.com/libjpeg-turbo/libjpeg-turbo.git libjpeg-turbo-src
fi
# Build
PREFIX="$DIR/build/prefix"
mkdir -p "$DIR/build"
cmake -S libjpeg-turbo-src -B "$DIR/build" \
-DCMAKE_BUILD_TYPE=MinSizeRel \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=OFF \
-DCMAKE_INSTALL_PREFIX="$PREFIX" \
-DCMAKE_INSTALL_LIBDIR=lib \
-DENABLE_SHARED=OFF \
-DENABLE_STATIC=ON \
-DWITH_TURBOJPEG=OFF
cmake --build "$DIR/build" -j"$NJOBS"
cmake --install "$DIR/build"
# Copy to package install dir
rm -rf "$INSTALL_DIR"
mkdir -p "$INSTALL_DIR"/{lib,include}
# Library
cp "$PREFIX/lib/libjpeg.a" "$INSTALL_DIR/lib/"
# Headers
cp "$PREFIX/include/jpeglib.h" "$INSTALL_DIR/include/"
cp "$PREFIX/include/jconfig.h" "$INSTALL_DIR/include/"
cp "$PREFIX/include/jerror.h" "$INSTALL_DIR/include/"
cp "$PREFIX/include/jmorecfg.h" "$INSTALL_DIR/include/"
# Clean up
rm -rf libjpeg-turbo-src "$DIR/build"
echo "Installed libjpeg to $INSTALL_DIR"
du -sh "$INSTALL_DIR"

View File

@@ -0,0 +1,10 @@
import os
DIR = os.path.join(os.path.dirname(__file__), "install")
LIB_DIR = os.path.join(DIR, "lib")
INCLUDE_DIR = os.path.join(DIR, "include")
def smoketest():
assert os.path.isfile(os.path.join(LIB_DIR, "libjpeg.a")), "libjpeg.a not found"
assert os.path.isfile(os.path.join(INCLUDE_DIR, "jpeglib.h")), "jpeglib.h not found"

15
libjpeg/pyproject.toml Normal file
View File

@@ -0,0 +1,15 @@
[build-system]
requires = ["setuptools>=64", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "libjpeg"
version = "3.1.0"
description = "libjpeg-turbo JPEG library (static build)"
requires-python = ">=3.8"
[tool.setuptools.packages.find]
include = ["libjpeg*"]
[tool.setuptools.package-data]
libjpeg = ["install/**/*"]

61
libjpeg/setup.py Normal file
View File

@@ -0,0 +1,61 @@
import os
import platform
import subprocess
from setuptools.command.build_py import build_py
try:
from wheel.bdist_wheel import bdist_wheel
except ImportError:
bdist_wheel = None
class BuildLibjpeg(build_py):
"""Run build.sh to compile libjpeg-turbo before collecting package data."""
def run(self):
pkg_dir = os.path.dirname(os.path.abspath(__file__))
marker = os.path.join(pkg_dir, "libjpeg", "install", "lib", "libjpeg.a")
if not os.path.exists(marker):
build_script = os.path.join(pkg_dir, "build.sh")
subprocess.check_call(["bash", build_script], cwd=pkg_dir)
super().run()
cmdclass = {"build_py": BuildLibjpeg}
if bdist_wheel is not None:
class PlatformWheel(bdist_wheel):
"""Produce a platform-specific, Python-version-agnostic wheel."""
def finalize_options(self):
super().finalize_options()
self.root_is_pure = False
def get_tag(self):
system = platform.system()
machine = platform.machine()
if system == "Linux":
plat = f"linux_{machine}"
elif system == "Darwin":
plat = "macosx_11_0_arm64"
else:
plat = f"{system.lower()}_{machine}"
return "py3", "none", plat
cmdclass["bdist_wheel"] = PlatformWheel
def setup():
from setuptools import setup as _setup
_setup(cmdclass=cmdclass)
if __name__ == "__main__":
setup()

73
ncurses/build.sh Executable file
View File

@@ -0,0 +1,73 @@
#!/usr/bin/env bash
set -e
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
cd "$DIR"
VERSION="6.5"
INSTALL_DIR="$DIR/ncurses/install"
# Idempotent: skip if already built
if [ -f "$INSTALL_DIR/lib/libncurses.a" ]; then
echo "ncurses already present, skipping build."
exit 0
fi
NJOBS="$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 2)"
# Download
TARBALL="ncurses-${VERSION}.tar.gz"
if [ ! -f "$TARBALL" ]; then
curl -fsSL "https://ftp.gnu.org/gnu/ncurses/${TARBALL}" -o "$TARBALL"
fi
# Extract
if [ ! -d "ncurses-${VERSION}" ]; then
tar xf "$TARBALL"
fi
# Build
PREFIX="$DIR/build/prefix"
mkdir -p "$DIR/build"
cd "ncurses-${VERSION}"
./configure \
--prefix="$PREFIX" \
--without-shared \
--with-normal \
--without-debug \
--without-cxx \
--without-cxx-binding \
--without-ada \
--without-manpages \
--without-progs \
--without-tests \
--without-dlsym \
--enable-overwrite
make -j"$NJOBS"
# Only install libs and headers; skip terminfo database (fails on macOS CI)
make install.libs install.includes
cd "$DIR"
# Copy to package install dir
rm -rf "$INSTALL_DIR"
mkdir -p "$INSTALL_DIR"/{lib,include}
# Libraries (ncurses 6.x builds wide-char by default; provide as libncurses.a)
cp "$PREFIX/lib/libncursesw.a" "$INSTALL_DIR/lib/libncurses.a" 2>/dev/null \
|| cp "$PREFIX/lib/libncurses.a" "$INSTALL_DIR/lib/"
# Headers (--enable-overwrite puts them directly in include/)
cp "$PREFIX/include/ncurses.h" "$INSTALL_DIR/include/"
cp "$PREFIX/include/curses.h" "$INSTALL_DIR/include/"
cp "$PREFIX/include/ncurses_dll.h" "$INSTALL_DIR/include/"
cp "$PREFIX/include/unctrl.h" "$INSTALL_DIR/include/"
cp "$PREFIX/include/term.h" "$INSTALL_DIR/include/"
cp "$PREFIX/include/termcap.h" "$INSTALL_DIR/include/"
# Clean up
rm -rf "ncurses-${VERSION}" "$TARBALL" "$DIR/build"
echo "Installed ncurses to $INSTALL_DIR"
du -sh "$INSTALL_DIR"

View File

@@ -0,0 +1,10 @@
import os
DIR = os.path.join(os.path.dirname(__file__), "install")
LIB_DIR = os.path.join(DIR, "lib")
INCLUDE_DIR = os.path.join(DIR, "include")
def smoketest():
assert os.path.isfile(os.path.join(LIB_DIR, "libncurses.a")), "libncurses.a not found"
assert os.path.isfile(os.path.join(INCLUDE_DIR, "ncurses.h")), "ncurses.h not found"

15
ncurses/pyproject.toml Normal file
View File

@@ -0,0 +1,15 @@
[build-system]
requires = ["setuptools>=64", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "ncurses"
version = "6.5"
description = "ncurses terminal library (static build)"
requires-python = ">=3.8"
[tool.setuptools.packages.find]
include = ["ncurses*"]
[tool.setuptools.package-data]
ncurses = ["install/**/*"]

61
ncurses/setup.py Normal file
View File

@@ -0,0 +1,61 @@
import os
import platform
import subprocess
from setuptools.command.build_py import build_py
try:
from wheel.bdist_wheel import bdist_wheel
except ImportError:
bdist_wheel = None
class BuildNcurses(build_py):
"""Run build.sh to compile ncurses before collecting package data."""
def run(self):
pkg_dir = os.path.dirname(os.path.abspath(__file__))
marker = os.path.join(pkg_dir, "ncurses", "install", "lib", "libncurses.a")
if not os.path.exists(marker):
build_script = os.path.join(pkg_dir, "build.sh")
subprocess.check_call(["bash", build_script], cwd=pkg_dir)
super().run()
cmdclass = {"build_py": BuildNcurses}
if bdist_wheel is not None:
class PlatformWheel(bdist_wheel):
"""Produce a platform-specific, Python-version-agnostic wheel."""
def finalize_options(self):
super().finalize_options()
self.root_is_pure = False
def get_tag(self):
system = platform.system()
machine = platform.machine()
if system == "Linux":
plat = f"linux_{machine}"
elif system == "Darwin":
plat = "macosx_11_0_arm64"
else:
plat = f"{system.lower()}_{machine}"
return "py3", "none", plat
cmdclass["bdist_wheel"] = PlatformWheel
def setup():
from setuptools import setup as _setup
_setup(cmdclass=cmdclass)
if __name__ == "__main__":
setup()

74
openssl3/build.sh Executable file
View File

@@ -0,0 +1,74 @@
#!/usr/bin/env bash
set -e
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
cd "$DIR"
VERSION="3.4.1"
INSTALL_DIR="$DIR/openssl3/install"
# Idempotent: skip if already built
if [ -f "$INSTALL_DIR/lib/libcrypto.a" ]; then
echo "openssl already present, skipping build."
exit 0
fi
NJOBS="$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 2)"
# Download
TARBALL="openssl-${VERSION}.tar.gz"
if [ ! -d "openssl-src" ]; then
curl -fSL -o "$TARBALL" "https://github.com/openssl/openssl/releases/download/openssl-${VERSION}/${TARBALL}"
mkdir -p openssl-src
tar -xf "$TARBALL" -C openssl-src --strip-components=1
rm -f "$TARBALL"
fi
# Configure
PREFIX="$DIR/build/prefix"
cd openssl-src
if [ "$(uname)" = "Darwin" ]; then
TARGET="darwin64-arm64-cc"
else
MACHINE="$(uname -m)"
if [ "$MACHINE" = "x86_64" ]; then
TARGET="linux-x86_64"
elif [ "$MACHINE" = "aarch64" ]; then
TARGET="linux-aarch64"
else
TARGET="linux-${MACHINE}"
fi
fi
./Configure "$TARGET" \
--prefix="$PREFIX" \
--libdir=lib \
no-shared \
no-tests \
no-docs \
no-apps \
-Os
# Build
make -j"$NJOBS"
make install_sw
cd "$DIR"
# Copy to package install dir
rm -rf "$INSTALL_DIR"
mkdir -p "$INSTALL_DIR"/{lib,include}
# Libraries
cp "$PREFIX/lib/libcrypto.a" "$INSTALL_DIR/lib/"
cp "$PREFIX/lib/libssl.a" "$INSTALL_DIR/lib/"
# Headers
cp -r "$PREFIX/include/openssl" "$INSTALL_DIR/include/"
# Clean up
rm -rf openssl-src "$DIR/build"
echo "Installed openssl to $INSTALL_DIR"
du -sh "$INSTALL_DIR"

View File

@@ -0,0 +1,11 @@
import os
DIR = os.path.join(os.path.dirname(__file__), "install")
LIB_DIR = os.path.join(DIR, "lib")
INCLUDE_DIR = os.path.join(DIR, "include")
def smoketest():
assert os.path.isfile(os.path.join(LIB_DIR, "libcrypto.a")), "libcrypto.a not found"
assert os.path.isfile(os.path.join(LIB_DIR, "libssl.a")), "libssl.a not found"
assert os.path.isdir(os.path.join(INCLUDE_DIR, "openssl")), "openssl headers not found"

15
openssl3/pyproject.toml Normal file
View File

@@ -0,0 +1,15 @@
[build-system]
requires = ["setuptools>=64", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "openssl3"
version = "3.4.1"
description = "OpenSSL cryptography library (static build)"
requires-python = ">=3.8"
[tool.setuptools.packages.find]
include = ["openssl3*"]
[tool.setuptools.package-data]
openssl3 = ["install/**/*"]

61
openssl3/setup.py Normal file
View File

@@ -0,0 +1,61 @@
import os
import platform
import subprocess
from setuptools.command.build_py import build_py
try:
from wheel.bdist_wheel import bdist_wheel
except ImportError:
bdist_wheel = None
class BuildOpenssl(build_py):
"""Run build.sh to compile OpenSSL before collecting package data."""
def run(self):
pkg_dir = os.path.dirname(os.path.abspath(__file__))
marker = os.path.join(pkg_dir, "openssl3", "install", "lib", "libcrypto.a")
if not os.path.exists(marker):
build_script = os.path.join(pkg_dir, "build.sh")
subprocess.check_call(["bash", build_script], cwd=pkg_dir)
super().run()
cmdclass = {"build_py": BuildOpenssl}
if bdist_wheel is not None:
class PlatformWheel(bdist_wheel):
"""Produce a platform-specific, Python-version-agnostic wheel."""
def finalize_options(self):
super().finalize_options()
self.root_is_pure = False
def get_tag(self):
system = platform.system()
machine = platform.machine()
if system == "Linux":
plat = f"linux_{machine}"
elif system == "Darwin":
plat = "macosx_11_0_arm64"
else:
plat = f"{system.lower()}_{machine}"
return "py3", "none", plat
cmdclass["bdist_wheel"] = PlatformWheel
def setup():
from setuptools import setup as _setup
_setup(cmdclass=cmdclass)
if __name__ == "__main__":
setup()

44
python3-dev/build.sh Executable file
View File

@@ -0,0 +1,44 @@
#!/usr/bin/env bash
set -e
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
cd "$DIR"
VERSION="3.12.8"
INSTALL_DIR="$DIR/python3_dev/install"
# Idempotent: skip if already present
if [ -f "$INSTALL_DIR/include/Python.h" ]; then
echo "python3-dev headers already present, skipping."
exit 0
fi
TARBALL="Python-${VERSION}.tgz"
URL="https://www.python.org/ftp/python/${VERSION}/${TARBALL}"
echo "Downloading CPython ${VERSION} source ..."
curl -fSL -o "$TARBALL" "$URL"
echo "Extracting ..."
tar xzf "$TARBALL"
cd "Python-${VERSION}"
# Run configure to generate pyconfig.h for this platform
echo "Running configure to generate pyconfig.h ..."
./configure --disable-shared --without-ensurepip > /dev/null 2>&1
cd "$DIR"
# Copy headers
rm -rf "$INSTALL_DIR"
mkdir -p "$INSTALL_DIR/include"
cp -r "Python-${VERSION}/Include/"* "$INSTALL_DIR/include/"
cp "Python-${VERSION}/pyconfig.h" "$INSTALL_DIR/include/"
# Clean up
rm -rf "Python-${VERSION}" "$TARBALL"
echo "Installed python3-dev headers to $INSTALL_DIR"
du -sh "$INSTALL_DIR"

View File

@@ -0,0 +1,15 @@
[build-system]
requires = ["setuptools>=64", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "python3-dev"
version = "3.12.8"
description = "CPython development headers"
requires-python = ">=3.8"
[tool.setuptools.packages.find]
include = ["python3_dev*"]
[tool.setuptools.package-data]
python3_dev = ["install/**/*"]

View File

@@ -0,0 +1,9 @@
import os
DIR = os.path.join(os.path.dirname(__file__), "install")
INCLUDE_DIR = os.path.join(DIR, "include")
def smoketest():
assert os.path.isfile(os.path.join(INCLUDE_DIR, "Python.h")), "Python.h not found"
assert os.path.isfile(os.path.join(INCLUDE_DIR, "pyconfig.h")), "pyconfig.h not found"

61
python3-dev/setup.py Normal file
View File

@@ -0,0 +1,61 @@
import os
import platform
import subprocess
from setuptools.command.build_py import build_py
try:
from wheel.bdist_wheel import bdist_wheel
except ImportError:
bdist_wheel = None
class BuildPython3Dev(build_py):
"""Run build.sh to download CPython headers before collecting package data."""
def run(self):
pkg_dir = os.path.dirname(os.path.abspath(__file__))
marker = os.path.join(pkg_dir, "python3_dev", "install", "include", "Python.h")
if not os.path.exists(marker):
build_script = os.path.join(pkg_dir, "build.sh")
subprocess.check_call(["bash", build_script], cwd=pkg_dir)
super().run()
cmdclass = {"build_py": BuildPython3Dev}
if bdist_wheel is not None:
class PlatformWheel(bdist_wheel):
"""Produce a platform-specific, Python-version-agnostic wheel."""
def finalize_options(self):
super().finalize_options()
self.root_is_pure = False
def get_tag(self):
system = platform.system()
machine = platform.machine()
if system == "Linux":
plat = f"linux_{machine}"
elif system == "Darwin":
plat = "macosx_11_0_arm64"
else:
plat = f"{system.lower()}_{machine}"
return "py3", "none", plat
cmdclass["bdist_wheel"] = PlatformWheel
def setup():
from setuptools import setup as _setup
_setup(cmdclass=cmdclass)
if __name__ == "__main__":
setup()

58
zstd/build.sh Executable file
View File

@@ -0,0 +1,58 @@
#!/usr/bin/env bash
set -e
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
cd "$DIR"
VERSION="1.5.6"
INSTALL_DIR="$DIR/zstd/install"
# Idempotent: skip if already built
if [ -f "$INSTALL_DIR/lib/libzstd.a" ]; then
echo "zstd already present, skipping build."
exit 0
fi
NJOBS="$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 2)"
# Clone
if [ ! -d "zstd-src" ]; then
git clone --depth 1 --branch "v${VERSION}" https://github.com/facebook/zstd.git zstd-src
fi
# Build
PREFIX="$DIR/build/prefix"
mkdir -p "$DIR/build"
cmake -S zstd-src/build/cmake -B "$DIR/build" \
-DCMAKE_BUILD_TYPE=MinSizeRel \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=OFF \
-DCMAKE_INSTALL_PREFIX="$PREFIX" \
-DCMAKE_INSTALL_LIBDIR=lib \
-DCMAKE_POLICY_VERSION_MINIMUM=3.5 \
-DZSTD_BUILD_PROGRAMS=OFF \
-DZSTD_BUILD_TESTS=OFF \
-DZSTD_BUILD_CONTRIB=OFF \
-DZSTD_BUILD_SHARED=OFF \
-DZSTD_BUILD_STATIC=ON
cmake --build "$DIR/build" -j"$NJOBS"
cmake --install "$DIR/build"
# Copy to package install dir
rm -rf "$INSTALL_DIR"
mkdir -p "$INSTALL_DIR"/{lib,include}
# Library
cp "$PREFIX/lib/libzstd.a" "$INSTALL_DIR/lib/"
# Headers
cp "$PREFIX/include/zstd.h" "$INSTALL_DIR/include/"
cp "$PREFIX/include/zstd_errors.h" "$INSTALL_DIR/include/"
cp "$PREFIX/include/zdict.h" "$INSTALL_DIR/include/"
# Clean up
rm -rf zstd-src "$DIR/build"
echo "Installed zstd to $INSTALL_DIR"
du -sh "$INSTALL_DIR"

15
zstd/pyproject.toml Normal file
View File

@@ -0,0 +1,15 @@
[build-system]
requires = ["setuptools>=64", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "zstd"
version = "1.5.6"
description = "Zstandard compression library (static build)"
requires-python = ">=3.8"
[tool.setuptools.packages.find]
include = ["zstd*"]
[tool.setuptools.package-data]
zstd = ["install/**/*"]

61
zstd/setup.py Normal file
View File

@@ -0,0 +1,61 @@
import os
import platform
import subprocess
from setuptools.command.build_py import build_py
try:
from wheel.bdist_wheel import bdist_wheel
except ImportError:
bdist_wheel = None
class BuildZstd(build_py):
"""Run build.sh to compile zstd before collecting package data."""
def run(self):
pkg_dir = os.path.dirname(os.path.abspath(__file__))
marker = os.path.join(pkg_dir, "zstd", "install", "lib", "libzstd.a")
if not os.path.exists(marker):
build_script = os.path.join(pkg_dir, "build.sh")
subprocess.check_call(["bash", build_script], cwd=pkg_dir)
super().run()
cmdclass = {"build_py": BuildZstd}
if bdist_wheel is not None:
class PlatformWheel(bdist_wheel):
"""Produce a platform-specific, Python-version-agnostic wheel."""
def finalize_options(self):
super().finalize_options()
self.root_is_pure = False
def get_tag(self):
system = platform.system()
machine = platform.machine()
if system == "Linux":
plat = f"linux_{machine}"
elif system == "Darwin":
plat = "macosx_11_0_arm64"
else:
plat = f"{system.lower()}_{machine}"
return "py3", "none", plat
cmdclass["bdist_wheel"] = PlatformWheel
def setup():
from setuptools import setup as _setup
_setup(cmdclass=cmdclass)
if __name__ == "__main__":
setup()

10
zstd/zstd/__init__.py Normal file
View File

@@ -0,0 +1,10 @@
import os
DIR = os.path.join(os.path.dirname(__file__), "install")
LIB_DIR = os.path.join(DIR, "lib")
INCLUDE_DIR = os.path.join(DIR, "include")
def smoketest():
assert os.path.isfile(os.path.join(LIB_DIR, "libzstd.a")), "libzstd.a not found"
assert os.path.isfile(os.path.join(INCLUDE_DIR, "zstd.h")), "zstd.h not found"