diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index b5ec3ad6e..efad54ab8 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -43,30 +43,24 @@ java-buildpack/ ├── bin/ │ ├── compile # Legacy V2 API entrypoint │ ├── detect # Detection phase entrypoint +│ ├── release # Release phase entrypoint │ ├── finalize # Finalize phase entrypoint (V3) │ └── supply # Supply phase entrypoint (V3) │ -├── config/ # Component configurations -│ ├── components.yml # Component registry -│ ├── cache.yml # Caching configuration -│ ├── repository.yml # Dependency repository config -│ └── *.yml # Individual component configs -│ -├── resources/ # Static resources for components -│ ├── tomcat/ # Tomcat configuration templates -│ ├── protect_app_security_provider/ -│ └── ... -│ +├── src/integration/ # Integraion tests ├── src/java/ # Go source code +| ├── common/ # Common libbuildpack integrations │ ├── containers/ # Container implementations │ ├── frameworks/ # Framework implementations │ ├── jres/ # JRE implementations +| ├── resources/ # Resource configuration files │ ├── supply/ # Supply phase orchestration │ │ └── cli/ # Supply CLI entrypoint │ └── finalize/ # Finalize phase orchestration │ └── cli/ # Finalize CLI entrypoint │ ├── docs/ # Documentation +├── ci/ # CI scripts └── scripts/ # Build and test scripts ``` @@ -139,7 +133,7 @@ The buildpack uses three main component types: ## Buildpack Lifecycle -The buildpack follows Cloud Foundry's V3 lifecycle with three phases: +The buildpack follows Cloud Foundry's V3 lifecycle with four phases: ### 1. Detect Phase @@ -171,7 +165,7 @@ The buildpack follows Cloud Foundry's V3 lifecycle with three phases: **Flow**: ``` -1. Load component registry (config/components.yml) +1. Load component registries for containers, jres and frameworks 2. For each component type (JRE, Frameworks): a. Run Detect() method b. If detected, run Supply() method @@ -222,6 +216,20 @@ The buildpack follows Cloud Foundry's V3 lifecycle with three phases: - Generates runtime configuration - Profile.d scripts run before app launch +### 4. Release Phase + +**Purpose**: Assemble the start command that the Cloud Controller wil use to start +the running java process of the application. + +**Entry Point**: `bin/release` + +**Flow**: +``` +Output the launch command written previouly in the finalize phase to $BUILD_DIR/tmp/java-buildpack-release-step.yml + +Note that the java-buildpack-release-step.yml follows strict predefined structure which is expected from CC. +If anything except this yaml appears in release output, the staged application will fail to start. +``` --- ## Key Architectural Patterns @@ -436,44 +444,6 @@ export JAVA_OPTS="${JAVA_OPTS} -Dnewrelic.config.license_key=%s" --- -## Configuration System - -### Component Registry - -**File**: `config/components.yml` - -**Purpose**: Declare available components - -**Structure**: -```yaml -containers: - - "JavaBuildpack::Container::SpringBoot" - - "JavaBuildpack::Container::Tomcat" - # ... - -jres: - - "JavaBuildpack::Jre::OpenJdkJRE" - # ... - -frameworks: - - "JavaBuildpack::Framework::NewRelicAgent" - - "JavaBuildpack::Framework::JavaOpts" - # ... -``` - -### Component Configuration - -**Pattern**: `config/.yml` - -**Purpose**: Configure individual components - -**Example** (`config/new_relic_agent.yml`): -```yaml -version: 8.7.+ -repository_root: https://download.run.pivotal.io/new-relic -enabled: true -``` - ### Environment Variable Overrides Users can override configuration via environment variables: @@ -573,12 +543,12 @@ Scripts in `/.profile.d/` run before app launch: ### Supply Phase Order 1. **JREs** - Install Java runtime first -2. **Frameworks** - Process in `components.yml` order +2. **Frameworks** - Process in `registry.RegisterStandardFrameworks()` order ### Finalize Phase Order 1. **JRE** - Configure Java runtime -2. **Frameworks** - Process in `components.yml` order +2. **Frameworks** - Process in `registry.RegisterStandardFrameworks()` order 3. **Container** - Generate launch command (last) **Important**: `JavaOpts` framework must be last to allow user overrides diff --git a/README.md b/README.md index b00d2fdec..a4958d427 100644 --- a/README.md +++ b/README.md @@ -123,8 +123,7 @@ The buildpack supports extension through the use of Git repository forking. The This Go-based buildpack is a migration from the original Ruby-based Cloud Foundry Java Buildpack. For comprehensive information about the migration status, component parity, and architectural differences: -* **[Ruby vs Go Buildpack Comparison](comparison.md)** - Comprehensive comparison of components, features, and production readiness assessment (92.9% component parity, production-ready for 98%+ of Java applications) -* **[Dependency Installation Comparison](ruby_vs_go_buildpack_comparison.md)** - Technical deep-dive into how dependency extraction differs between Ruby and Go implementations +* **[Ruby vs Go Buildpack Comparison](ruby_vs_go_buildpack_comparison.md)** - Technical deep-dive into how dependency extraction differs between Ruby and Go implementations **⚠️ Important Migration Note:** The Go buildpack does **NOT** support the Ruby buildpack's `repository_root` configuration approach for custom JREs (via `JBP_CONFIG_*` environment variables). Custom JREs now require forking the buildpack and modifying `manifest.yml`. See [Custom JRE Usage](docs/custom-jre-usage.md) for details. @@ -197,16 +196,6 @@ For historical analysis documents from development sessions, see [`docs/archive/ * [GraalVM](docs/jre-graal_vm_jre.md) ([Configuration](docs/jre-graal_vm_jre.md#configuration)) * [IBM Semeru](docs/jre-ibm_jre.md) ([Configuration](docs/jre-ibm_jre.md#configuration)) * [Oracle](docs/jre-oracle_jre.md) ([Configuration](docs/jre-oracle_jre.md#configuration)) -* [Extending](docs/extending.md) - * [Application](docs/extending-application.md) - * [Droplet](docs/extending-droplet.md) - * [BaseComponent](docs/extending-base_component.md) - * [VersionedDependencyComponent](docs/extending-versioned_dependency_component.md) - * [ModularComponent](docs/extending-modular_component.md) - * [Caches](docs/extending-caches.md) ([Configuration](docs/extending-caches.md#configuration)) - * [Logging](docs/extending-logging.md) ([Configuration](docs/extending-logging.md#configuration)) - * [Repositories](docs/extending-repositories.md) ([Configuration](docs/extending-repositories.md#configuration)) - * [Utilities](docs/extending-utilities.md) * [Debugging the Buildpack](docs/debugging-the-buildpack.md) * [Buildpack Modes](docs/buildpack-modes.md) * Related Projects @@ -320,6 +309,7 @@ $ ./scripts/unit.sh $ BUILDPACK_FILE="$(pwd)/build/buildpack.zip" \ ./scripts/integration.sh --platform docker --parallel true --github-token MYTOKEN ``` +For detailed guidelines about setting up and running tests please check this [Testing Guide](docs/TESTING.md) [Running Cloud Foundry locally][] is useful for privately testing new features. diff --git a/comparison.md b/comparison.md deleted file mode 100644 index 568aecac5..000000000 --- a/comparison.md +++ /dev/null @@ -1,663 +0,0 @@ -# Java Buildpack: Ruby vs Go Implementation Comparison - -**Date**: December 16, 2025 -**Migration Status**: ~95% Complete -**Last Commit**: 4527918f (Framework bug fixes and enhancements) -**Test Status**: All integration tests passing ✅ - ---- - -## Executive Summary - -The Go-based Java buildpack migration has achieved **feature parity** with the Ruby implementation for **mainstream Java applications**. This document provides a comprehensive comparison of components, configuration mechanisms, and identifies the remaining gaps. - -### Migration Progress - -| Category | Ruby Files | Go Files | Completion | Status | -|----------|-----------|----------|------------|--------| -| **Containers** | 9 | 8 (+utils) | 100% | ✅ Complete | -| **Frameworks** | 40 | 37 | 92.5% | ✅ Near Complete | -| **JREs** | 7 | 7 | 100% | ✅ Complete | -| **Total Components** | 56 | 52 | 92.9% | ✅ Production Ready | - -### Key Findings - -**✅ PRODUCTION READY** for: -- Spring Boot, Tomcat, and Jakarta EE applications (100% coverage) -- All major Java container types (Groovy, Java Main, Play, Ratpack) -- All 7 JRE providers (OpenJDK, Zulu, SAP Machine, GraalVM, IBM, Oracle, Zing) -- 15 major APM/monitoring agents (New Relic, AppDynamics, Dynatrace, etc.) -- Common profilers (JProfiler, YourKit, JaCoCo) -- Database auto-injection (PostgreSQL, MariaDB) -- Spring auto-reconfiguration and Cloud Foundry integration -- Security providers (Luna HSM, ProtectApp, Seeker, Container Security) -- Custom container customizer scripts and Java Memory Assistant - -**⚠️ EVALUATE CAREFULLY** for: -- Organizations requiring legacy/deprecated frameworks (Spring Insight) -- Multi-buildpack coordination scenarios - ---- - -## 1. Container Implementations (100% Complete) - -### 1.1 Fully Migrated Containers - -All 8 container types from Ruby have been successfully migrated to Go: - -| Container | Ruby File | Go File | Integration Tests | Notes | -|-----------|-----------|---------|------------------|-------| -| **Spring Boot** | `spring_boot.rb` | `spring_boot.go` | ✅ 5 tests | Detects embedded servers, manifest entries | -| **Tomcat** | `tomcat.rb` | `tomcat.go` | ✅ 5 tests | WAR deployment, servlet containers | -| **Spring Boot CLI** | `spring_boot_cli.rb` | `spring_boot_cli.go` | ✅ 6 tests | Spring Boot CLI script execution | -| **Groovy** | `groovy.rb` | `groovy.go` + `groovy_utils.go` | ✅ 5 tests | Groovy script execution | -| **Java Main** | `java_main.rb` | `java_main.go` | ✅ 4 tests | Main-Class manifest applications | -| **Play Framework** | `play_framework.rb` | `play.go` | ✅ 8 tests | Play 2.x staged & dist modes | -| **Dist ZIP** | `dist_zip.rb` + `dist_zip_like.rb` | `dist_zip.go` | ✅ 4 tests | Distribution archives | -| **Ratpack** | `ratpack.rb` | Merged into `dist_zip.go` | ✅ 3 tests | Unified with Dist ZIP | - -**Total**: 40 integration tests covering all containers (all passing) - -### 1.2 Architecture Improvements - -**Ratpack/DistZip Unification**: -- Ruby had 3 separate files: `dist_zip_like.rb` (base), `dist_zip.rb`, `ratpack.rb` -- Go unified into single `dist_zip.go` (231 lines) - cleaner architecture -- Detects both Dist ZIP and Ratpack applications with shared logic - -**Container Detection Order** (critical for correct app type identification): -``` -Spring Boot → Tomcat → Spring Boot CLI → Groovy → Play → DistZip → Java Main → Ratpack -``` - ---- - -## 2. Framework Implementations (82.5% Complete) - -### 2.1 Fully Migrated Frameworks (37 frameworks) - -#### APM & Monitoring Agents (15 frameworks) ✅ - -| Framework | Ruby File | Go File | Tests | Status | -|-----------|-----------|---------|-------|--------| -| New Relic | `new_relic_agent.rb` | `new_relic.go` | 2 | ✅ Complete | -| AppDynamics | `app_dynamics_agent.rb` | `app_dynamics.go` | 2 | ✅ Complete | -| Dynatrace OneAgent | `dynatrace_one_agent.rb` | `dynatrace.go` | 2 | ✅ Complete | -| Azure App Insights | `azure_application_insights_agent.rb` | `azure_application_insights_agent.go` | 2 | ✅ Complete | -| Datadog | `datadog_javaagent.rb` | `datadog_javaagent.go` | 2 | ✅ Complete | -| Elastic APM | `elastic_apm_agent.rb` | `elastic_apm_agent.go` | 2 | ✅ Complete | -| SkyWalking | `sky_walking_agent.rb` | `sky_walking_agent.go` | 2 | ✅ Complete | -| Splunk OTEL | `splunk_otel_java_agent.rb` | `splunk_otel_java_agent.go` | 2 | ✅ Complete | -| OpenTelemetry | `open_telemetry_javaagent.rb` | `open_telemetry_javaagent.go` | 2 | ✅ Complete | -| Checkmarx IAST | `checkmarx_iast_agent.rb` | `checkmarx_iast_agent.go` | 1 | ✅ Complete | -| Contrast Security | `contrast_security_agent.rb` | `contrast_security_agent.go` | 0 | ✅ Complete | -| Introscope (CA APM) | `introscope_agent.rb` | `introscope_agent.go` | 0 | ✅ Complete | -| Riverbed AppInternals | `riverbed_appinternals_agent.rb` | `riverbed_appinternals_agent.go` | 0 | ✅ Complete | -| Google Stackdriver Profiler | `google_stackdriver_profiler.rb` | `google_stackdriver_profiler.go` | 1 | ✅ Complete | -| ~~Google Stackdriver Debugger~~ | `google_stackdriver_debugger.rb` | ~~`google_stackdriver_debugger.go`~~ | 0 | ❌ Removed (deprecated by Google) | - -#### Profiling & Code Coverage (6 frameworks) ✅ - -| Framework | Ruby File | Go File | Tests | Status | -|-----------|-----------|---------|-------|--------| -| JProfiler | `jprofiler_profiler.rb` | `jprofiler_profiler.go` | 0 | ✅ Complete | -| YourKit | `your_kit_profiler.rb` | `your_kit_profiler.go` | 0 | ✅ Complete | -| JaCoCo | `jacoco_agent.rb` | `jacoco_agent.go` | 1 | ✅ Complete | -| JRebel | `jrebel_agent.rb` | `jrebel_agent.go` | 0 | ✅ Complete | -| AspectJ Weaver | `aspectj_weaver_agent.rb` | `aspectj_weaver_agent.go` | 0 | ✅ Complete | -| Sealights | `sealights_agent.rb` | `sealights_agent.go` | 0 | ✅ Complete | - -#### Utility Frameworks (5 frameworks) ✅ - -| Framework | Ruby File | Go File | Tests | Status | -|-----------|-----------|---------|-------|--------| -| Debug (JDWP) | `debug.rb` | `debug.go` | 1 | ✅ Complete | -| JMX | `jmx.rb` | `jmx.go` | 1 | ✅ Complete | -| Java Opts | `java_opts.rb` | `java_opts.go` | 0 | ✅ Complete | -| Spring Auto Reconfig | `spring_auto_reconfiguration.rb` | `spring_auto_reconfiguration.go` | 1 | ✅ Complete | -| Java CF Env | `java_cf_env.rb` | `java_cf_env.go` | 1 | ✅ Complete | - -#### Database Drivers (2 frameworks) ✅ - -| Framework | Ruby File | Go File | Tests | Status | -|-----------|-----------|---------|-------|--------| -| PostgreSQL JDBC | `postgresql_jdbc.rb` | `postgresql_jdbc.go` | 1 | ✅ Complete | -| MariaDB JDBC | `maria_db_jdbc.rb` | `maria_db_jdbc.go` | 1 | ✅ Complete | - -#### Security & Certificates (6 frameworks) ✅ - -| Framework | Ruby File | Go File | Tests | Status | -|-----------|-----------|---------|-------|--------| -| Client Certificate Mapper | `client_certificate_mapper.rb` | `client_certificate_mapper.go` | 0 | ✅ Complete | -| Container Security Provider | `container_security_provider.rb` | `container_security_provider.go` | 0 | ✅ Complete | -| Luna Security Provider | `luna_security_provider.rb` | `luna_security_provider.go` | 0 | ✅ Complete | -| ProtectApp Security Provider | `protect_app_security_provider.rb` | `protect_app_security_provider.go` | 0 | ✅ Complete | -| Seeker Security Provider | `seeker_security_provider.rb` | `seeker_security_provider.go` | 0 | ✅ Complete | -| Container Customizer | `container_customizer.rb` | `container_customizer.go` | 0 | ✅ Complete | - -#### Other Utility Frameworks (2 frameworks) ✅ - -| Framework | Ruby File | Go File | Tests | Status | -|-----------|-----------|---------|-------|--------| -| Java Memory Assistant | `java_memory_assistant.rb` | `java_memory_assistant.go` | 0 | ✅ Complete | -| Metric Writer | `metric_writer.rb` | `metric_writer.go` | 0 | ✅ Complete | - -### 2.2 Recently Added Frameworks (December 2025) - -The following frameworks were added since the initial migration: - -| Framework | Ruby File | Go File | Status | Added | -|-----------|-----------|---------|--------|-------| -| **Container Customizer** | `container_customizer.rb` | `container_customizer.go` | ✅ Complete | Dec 2025 | -| **Java Memory Assistant** | `java_memory_assistant.rb` | `java_memory_assistant.go` | ✅ Complete | Dec 2025 | -| **Metric Writer** | `metric_writer.rb` | `metric_writer.go` | ✅ Complete | Dec 2025 | -| **ProtectApp Security Provider** | `protect_app_security_provider.rb` | `protect_app_security_provider.go` | ✅ Complete | Dec 2025 | -| **Seeker Security Provider** | `seeker_security_provider.rb` | `seeker_security_provider.go` | ✅ Complete | Dec 2025 | - -### 2.3 Missing Frameworks (3 frameworks - 7.5%) - -#### Not Migrated (Low Priority) - -| Framework | Ruby File | Priority | Reason | -|-----------|-----------|----------|--------| -| **Java Security** | `java_security.rb` | LOW | Custom security policies, rarely used | -| **Multi Buildpack** | `multi_buildpack.rb` | MEDIUM | Multi-buildpack is now default within the libbuildpack architecture | -| **Spring Insight** | `spring_insight.rb` | LOW | Legacy monitoring (replaced by modern APM) | - -### 2.4 Removed Frameworks - -| Framework | Ruby File | Reason for Removal | -|-----------|-----------|-------------------| -| **Takipi Agent (OverOps)** | `takipi_agent.rb` | Moved behind licensed login wall (Dec 2025) | - -**Note**: Missing frameworks represent niche, deprecated, or rarely-used use cases. The 37 implemented frameworks cover 98%+ of production Java applications. - ---- - -## 3. JRE Implementations (100% Complete) - -### 3.1 JRE Providers - -#### Included in Default Manifest (3 JREs) ✅ - -| JRE | Ruby File | Go File | Versions Supported | Default | Status | -|-----|-----------|---------|---------------------------------------------|---------|--------| -| **OpenJDK** | `open_jdk_jre.rb` | `openjdk.go` | 8, 11, 17, 21, 23 | 17.x | ✅ Complete | -| **Zulu (Azul)** | `zulu_jre.rb` | `zulu.go` | 8, 11, 17 | 11.x | ✅ Complete | -| **SAP Machine** | `sap_machine_jre.rb` | `sapmachine.go` | 11(ruby only), 17, 21(go only), 25(go only) | 21.x | ✅ Complete | - -#### BYOL JREs - Require Custom Manifest (4 JREs) ✅ - -These JREs are fully implemented but require users to fork the buildpack and add their own manifest entries due to licensing restrictions. See [Custom JRE Usage Guide](docs/custom-jre-usage.md). - -| JRE | Ruby File | Go File | Status | Notes | -|-----|-----------|---------|--------|-------| -| **GraalVM** | `graal_vm_jre.rb` | `graalvm.go` | ✅ Complete | Requires user-provided repository | -| **IBM Semeru** | `ibm_jre.rb` | `ibm.go` | ✅ Complete | Formerly IBM JRE, requires user-provided repository | -| **Oracle JRE** | `oracle_jre.rb` | `oracle.go` | ✅ Complete | Requires Oracle license & repository | -| **Zing JRE** | `zing_jre.rb` | `zing.go` | ✅ Complete | Requires Azul license & repository | - -### 3.2 JRE Components (All Migrated) ✅ - -| Component | Ruby File | Go File | Purpose | Status | -|-----------|-----------|---------|---------|--------| -| **JVMKill Agent** | `jvmkill_agent.rb` | `jvmkill.go` | OOM killer with heap dumps | ✅ Complete | -| **Memory Calculator** | `open_jdk_like_memory_calculator.rb` | `memory_calculator.go` | Runtime JVM memory tuning | ✅ Complete | - -**All JREs include**: -- JVMKill agent (OOM protection with heap dump generation) -- Memory Calculator (automatic JVM heap/stack sizing) -- JAVA_HOME environment setup -- Supply and Finalize lifecycle phases - -**BYOL JREs Note** (December 2025): -- GraalVM, IBM Semeru, Oracle, and Zing JREs were removed from the default `manifest.yml` to reduce confusion -- These JREs require user-provided licenses and repositories -- Implementations remain fully functional - users can enable them by forking the buildpack and adding manifest entries -- **IMPORTANT**: The Ruby buildpack's `repository_root` configuration approach (via `JBP_CONFIG_*` env vars) is **NOT supported** in Go -- Users **must** fork the buildpack and add JRE entries to `manifest.yml` - runtime repository configuration is not available -- See comprehensive guide: [Custom JRE Usage](docs/custom-jre-usage.md) - ---- - -## 4. Configuration Mechanisms - -### 4.1 Environment Variable Patterns - -Both Ruby and Go buildpacks support the **same configuration patterns**: - -| Pattern | Scope | Example | Purpose | -|---------|-------|---------|---------| -| `JBP_CONFIG_` | Application | `JBP_CONFIG_OPEN_JDK_JRE='{jre: {version: 11.+}}'` | Override component config | -| `JBP_DEFAULT_` | Platform | `JBP_DEFAULT_OPEN_JDK_JRE='{jre: {version: 11.+}}'` | Foundation-wide defaults | -| `JBP_CONFIG_COMPONENTS` | Application | `JBP_CONFIG_COMPONENTS='{jres: ["JavaBuildpack::Jre::ZuluJRE"]}'` | Select components | - -**Configuration Files**: Both use identical YAML configuration: -- 53 config files in `config/*.yml` (same in both Ruby and Go) -- Components: `config/components.yml` (defines active containers/frameworks/JREs) -- Each component has its own config file (e.g., `config/tomcat.yml`, `config/new_relic_agent.yml`) - -### 4.2 Configuration Compatibility - -The Go buildpack maintains **near 100% backward compatibility** with Ruby buildpack configuration: - -```bash -# Works in both Ruby and Go buildpacks -cf set-env my-app JBP_CONFIG_OPEN_JDK_JRE '{ jre: { version: 11.+ }, memory_calculator: { stack_threads: 25 } }' -cf set-env my-app JBP_CONFIG_TOMCAT '{ tomcat: { version: 10.1.+ } }' -cf set-env my-app JBP_CONFIG_NEW_RELIC_AGENT '{ enabled: true }' -``` - -### 4.3 Key Configuration Differences (Ruby vs Go) - -| Feature | Ruby Buildpack | Go Buildpack | Migration Impact | -|---------|---------------|--------------|------------------| -| **JRE `repository_root`** | ✅ Supported via `JBP_CONFIG_*` | ❌ **NOT Supported** | **Breaking Change** - Must fork buildpack and modify `manifest.yml` | -| **Component selection** | ✅ Via env vars | ✅ Via env vars | Compatible | -| **Version wildcards** | ✅ Supported (e.g., `11.+`) | ✅ Supported | Compatible | -| **CNB conventions** | ❌ Not supported | ✅ Supported (`BP_*`, `BPL_*`) | Go enhancement | - -**Critical Migration Note for BYOL JREs:** - -The Ruby buildpack allowed runtime configuration of custom JRE repositories: -```bash -# ❌ This worked in Ruby, but DOES NOT WORK in Go -cf set-env myapp JBP_CONFIG_ORACLE_JRE '{ jre: { repository_root: "https://my-repo.com/oracle" } }' -cf set-env myapp JBP_CONFIG_GRAAL_VM_JRE '{ jre: { repository_root: "https://my-repo.com/graalvm" } }' -``` - -The Go buildpack requires explicit manifest entries: -```yaml -# ✅ Required approach: Fork buildpack and add to manifest.yml -dependencies: - - name: oracle - version: 17.0.13 - uri: https://my-repo.com/oracle/jdk-17.0.13_linux-x64_bin.tar.gz - sha256: abc123... -``` - -This change improves security (SHA256 verification) and build reproducibility, but requires buildpack forking for custom JREs. - ---- - -## 5. Testing Coverage - -### 5.1 Integration Tests (BRATS) - -**Status**: All integration tests migrated and passing ✅ - -| Test Category | Tests | Status | Coverage | -|--------------|-------|--------|----------| -| Tomcat | 5 | ✅ Passing | WAR deployment, context.xml, versions | -| Spring Boot | 5 | ✅ Passing | Embedded servers, fat JARs, versions | -| Play Framework | 8 | ✅ Passing | Staged mode, dist mode, versions | -| Groovy | 5 | ✅ Passing | Scripts, Grape, versions | -| Java Main | 4 | ✅ Passing | Main-Class, classpath, versions | -| Spring Boot CLI | 6 | ✅ Passing | CLI scripts, versions | -| Dist ZIP & Ratpack | 7 | ✅ Passing | Archives, Ratpack apps, versions | -| **APM Frameworks** | 20 | ✅ Passing | Agent injection, VCAP_SERVICES | -| **Database Drivers** | 2 | ✅ Passing | JDBC auto-injection | -| **Utilities** | 4 | ✅ Passing | Debug, JMX, auto-reconfig | -| **Offline Mode** | 4 | ✅ Passing | Package cache, offline buildpack | - -**Total**: 70+ integration tests (all passing) - -### 5.2 Test Fixtures Migration - -**Status**: Complete migration from Ruby fixtures to Go structure ✅ - -| Category | Ruby Location | Go Location | Status | -|----------|--------------|-------------|--------| -| Container Apps | `spec/fixtures/container_*` | `src/integration/testdata/apps/` | ✅ Migrated | -| Framework Apps | `spec/fixtures/framework_*` | `src/integration/testdata/frameworks/` | ✅ Migrated | -| JRE Tests | `spec/fixtures/integration_*` | `src/integration/testdata/containers/` | ✅ Migrated | - ---- - -## 6. Packaging & Distribution - -### 6.1 Buildpack Structure - -Both Ruby and Go buildpacks produce identical buildpack archives: - -| Component | Ruby Buildpack | Go Buildpack | Notes | -|-----------|---------------|--------------|-------| -| **bin/detect** | Ruby script | Go binary | Container type detection | -| **bin/supply** | Ruby script | Go binary | Dependency installation | -| **bin/finalize** | Ruby script | Go binary | Final configuration | -| **bin/release** | Ruby script | Go binary | Process type generation | -| **config/*.yml** | 53 files | 53 files | Identical configuration | -| **resources/** | Templates, configs | Templates, configs | Identical resources | - -### 6.2 Online vs Offline Buildpacks - -Both Ruby and Go support **online** and **offline** modes: - -**Online Mode**: -- Downloads dependencies from buildpack manifest repository at staging time -- Smaller buildpack size (~1 MB) -- Requires internet access during staging - -**Offline Mode**: -- All dependencies pre-packaged in buildpack -- Larger buildpack size (~200-300 MB depending on cached dependencies) -- No internet access required during staging - -**Packaging**: -```bash -# Ruby buildpack -bundle exec rake package OFFLINE=true - -# Go buildpack -./scripts/package.sh -``` - ---- - -## 7. Key Architectural Differences - -### 7.1 Implementation Language - -| Aspect | Ruby Buildpack | Go Buildpack | -|--------|---------------|--------------| -| **Language** | Ruby 2.x-3.x | Go 1.25.4 | -| **Files** | 144 .rb files | 70 .go files | -| **Lines of Code** | ~15,000 LOC | ~8,000 LOC | -| **Dependencies** | Bundler, Ruby gems | None (static binary) | -| **Startup Time** | ~2-3s (Ruby VM overhead) | ~500ms (native binary) | -| **Memory Usage** | ~50-80 MB (Ruby VM) | ~20-30 MB (native binary) | - -### 7.2 Component Architecture - -**Ruby**: -- Object-oriented with inheritance (base classes: `BaseComponent`, `VersionedDependencyComponent`) -- Mixins for shared behavior -- Dynamic component loading via `components.yml` - -**Go**: -- Interface-based with composition -- No inheritance, explicit interfaces -- Static component registration - -**Both architectures support**: -- Pluggable components (containers, frameworks, JREs) -- Lifecycle phases (detect, supply, finalize, release) -- Configuration overrides via environment variables - -### 7.3 Dependency Extraction - -**Key Finding**: The Go implementation lost Ruby's automatic directory stripping during extraction. - -**Ruby**: -```ruby -shell "tar xzf #{file.path} -C #{@droplet.sandbox} --strip 1" -``` - -**Go**: -```go -// Extracts with nested directory, requires findTomcatHome() helper -dependency.Extract(tarball, targetDir) -tomcatHome := findTomcatHome(targetDir) // Workaround -``` - -**Impact**: Go buildpack requires additional helper functions (`findTomcatHome`, `findGroovyHome`) that weren't needed in Ruby. - -**Recommendation**: Enhance Go dependency extraction to support `--strip 1` equivalent behavior. - ---- - -## 8. Production Readiness Assessment - -### 8.1 Ready for Production ✅ - -The Go buildpack is **production-ready** for organizations using: - -**Application Types** (100% coverage): -- Spring Boot applications (most common - 60%+ of Java apps) -- Tomcat/Jakarta EE applications -- Groovy applications -- Java Main applications -- Play Framework applications -- Ratpack applications -- Dist ZIP applications - -**JRE Providers** (100% implementation, 3 included + 4 BYOL): -- OpenJDK (default, most common) - ✅ Included in manifest -- Azul Zulu (Azure-preferred) - ✅ Included in manifest -- SAP Machine (SAP shops) - ✅ Included in manifest -- GraalVM (native image support) - 🔧 BYOL via custom manifest (see [Custom JRE Guide](docs/custom-jre-usage.md)) -- IBM Semeru (IBM shops) - 🔧 BYOL via custom manifest -- Oracle JRE (Oracle customers) - 🔧 BYOL via custom manifest -- Azul Zing (ultra-low latency) - 🔧 BYOL via custom manifest - -**APM/Monitoring** (93% coverage): -- New Relic, AppDynamics, Dynatrace -- Azure App Insights, Datadog, Elastic APM -- SkyWalking, Splunk OTEL, OpenTelemetry -- Google Stackdriver (Debugger, Profiler) -- Contrast Security, Checkmarx IAST -- Introscope, Riverbed AppInternals - -**Profilers** (100% coverage): -- JProfiler, YourKit, JaCoCo -- JRebel, AspectJ Weaver -- Takipi/OverOps, Sealights - -**Database Auto-Injection** (100% coverage): -- PostgreSQL JDBC -- MariaDB JDBC - -### 8.2 Evaluate Carefully ⚠️ - -Organizations should **evaluate alternatives** if requiring: - -**Legacy/Deprecated Frameworks**: -- Spring Insight (deprecated, use modern APM) - Not yet implemented - -**Advanced Scenarios**: -- Multi-buildpack coordination (not yet implemented) -- Custom Java security policies (not yet implemented) - -### 8.3 Migration Path - -**For most organizations**: The Go buildpack is a **drop-in replacement** for the Ruby buildpack. - -**Steps**: -1. Update buildpack URL to point to Go buildpack repository -2. No application code changes required -3. No configuration changes required (same `JBP_CONFIG_*` variables) -4. Test staging and runtime behavior -5. Deploy to production - -**Rollback**: Keep Ruby buildpack available as fallback during transition period. - ---- - -## 9. Remaining Work - -### 9.3 Low Priority (2 items) - -1. **Java Security** - Custom security policies - -**Note**: Low-priority items represent <2% of production use cases. - -### 9.4 Documentation - -**Ruby buildpack documentation**: 75 markdown files in `docs/` - -**Go buildpack documentation**: Should create equivalent docs covering: -- Container-specific docs (12 files) -- Framework-specific docs (40 files) -- JRE-specific docs (7 files) -- General guides (extending, design, util, debugging) - -**Recommendation**: Migrate or link to Ruby buildpack docs until Go-specific docs are created. - ---- - -## 10. Performance Comparison - -| Metric | Ruby Buildpack | Go Buildpack | Improvement | -|--------|---------------|--------------|-------------| -| **Detect Phase** | ~2-3s | ~500ms | 4-6x faster | -| **Supply Phase** | ~20-30s | ~15-20s | 25-33% faster | -| **Finalize Phase** | ~3-5s | ~2-3s | 33-40% faster | -| **Memory Usage** | ~50-80 MB | ~20-30 MB | 50-60% reduction | -| **Buildpack Size** | ~15 MB (online) | ~5.5 MB (online) | reduction | -| **Offline Package** | ~1.6GB MB | ~1.1 GB | reduction | - -**Key Performance Benefits**: -- Native binary execution (no Ruby VM overhead) -- Faster startup times for detect/finalize phases -- Lower memory footprint during staging -- Identical download sizes and caching behavior - ---- - -## 11. Conclusion - -### 11.1 Migration Success - -The Go-based Java buildpack migration has achieved **92.9% component parity** and **100% coverage** for mainstream Java applications. The remaining 3 missing frameworks (7.5%) represent niche or deprecated use cases affecting <2% of production deployments. - -Recent additions (December 2025): -- Container Customizer -- Java Memory Assistant -- Metric Writer -- ProtectApp Security Provider -- Seeker Security Provider - -### 11.2 Recommendation - -**Deploy to production** for: -- Spring Boot microservices (most common use case) -- Tomcat/Jakarta EE applications -- Standard Java applications with APM monitoring -- Applications using mainstream JREs (OpenJDK, Zulu, SAP Machine) -- Applications requiring security providers (Luna, ProtectApp, Seeker) -- Applications using container customizers or memory assistant - -**Defer migration** only if: -- Requiring deprecated Spring Insight framework -- Using multi-buildpack setups (wait for multi-buildpack implementation) -- Requiring custom Java security policies - -### 11.3 Next Steps - -1. **Complete remaining 3 frameworks** (optional, based on user demand): - - Multi-buildpack coordination (medium priority) - - Java Security (low priority) - - Spring Insight (deprecated, low priority) -2. **Update documentation** to reflect new framework additions -3. **Performance testing** at scale (validate 4-6x faster detect phase) -4. **User acceptance testing** with pilot deployments -5. **Gradual rollout** to production with Ruby buildpack as fallback - ---- - -## Appendix A: Component Reference Tables - -### A.1 Containers (8 containers) - -| # | Container | Ruby File | Go File | Lines (Go) | Tests | -|---|-----------|-----------|---------|------------|-------| -| 1 | Spring Boot | `spring_boot.rb` | `spring_boot.go` | 197 | 5 | -| 2 | Tomcat | `tomcat.rb` | `tomcat.go` | 380 | 5 | -| 3 | Spring Boot CLI | `spring_boot_cli.rb` | `spring_boot_cli.go` | 213 | 6 | -| 4 | Groovy | `groovy.rb` | `groovy.go` + `groovy_utils.go` | 176 + 145 | 5 | -| 5 | Java Main | `java_main.rb` | `java_main.go` | 181 | 4 | -| 6 | Play Framework | `play_framework.rb` | `play.go` | 237 | 8 | -| 7 | Dist ZIP | `dist_zip.rb` + `dist_zip_like.rb` | `dist_zip.go` | 231 | 4 | -| 8 | Ratpack | `ratpack.rb` | Merged into `dist_zip.go` | (unified) | 3 | - -### A.2 JREs (7 JREs - 3 in manifest, 4 BYOL) - -| # | JRE | Ruby File | Go File | Lines (Go) | In Manifest | Notes | -|---|-----|-----------|---------|------------|-------------|-------| -| 1 | OpenJDK | `open_jdk_jre.rb` | `openjdk.go` | 138 | ✅ Yes | Default JRE (17.x) | -| 2 | Zulu | `zulu_jre.rb` | `zulu.go` | 142 | ✅ Yes | Azul Zulu (11.x default) | -| 3 | SAP Machine | `sap_machine_jre.rb` | `sapmachine.go` | 147 | ✅ Yes | SAP's OpenJDK (17.x default) | -| 4 | GraalVM | `graal_vm_jre.rb` | `graalvm.go` | 147 | 🔧 BYOL | User-configured via custom manifest | -| 5 | IBM Semeru | `ibm_jre.rb` | `ibm.go` | 150 | 🔧 BYOL | User-configured via custom manifest | -| 6 | Oracle JRE | `oracle_jre.rb` | `oracle.go` | 139 | 🔧 BYOL | User-configured via custom manifest | -| 7 | Zing JRE | `zing_jre.rb` | `zing.go` | 129 | 🔧 BYOL | User-configured via custom manifest | - -### A.3 Frameworks by Category - -**APM & Monitoring (14)**: -New Relic, AppDynamics, Dynatrace, Azure App Insights, Datadog, Elastic APM, SkyWalking, Splunk OTEL, OpenTelemetry, Checkmarx IAST, Contrast Security, Introscope, Riverbed AppInternals, Google Stackdriver Profiler - -**Profiling (6)**: -JProfiler, YourKit, JaCoCo, JRebel, AspectJ Weaver, Sealights - -**Utilities (7)**: -Debug (JDWP), JMX, Java Opts, Spring Auto Reconfiguration, Java CF Env, Java Memory Assistant, Metric Writer - -**Database (2)**: -PostgreSQL JDBC, MariaDB JDBC - -**Security & Container (7)**: -Client Certificate Mapper, Container Security Provider, Luna Security Provider, ProtectApp Security Provider, Seeker Security Provider, Container Customizer - ---- - -## Appendix B: Configuration Examples - -### B.1 JRE Selection - -```bash -# Use Zulu JRE instead of OpenJDK -cf set-env my-app JBP_CONFIG_COMPONENTS '{jres: ["JavaBuildpack::Jre::ZuluJRE"]}' - -# Use Java 11 -cf set-env my-app JBP_CONFIG_OPEN_JDK_JRE '{jre: {version: 11.+}}' - -# Adjust memory calculator -cf set-env my-app JBP_CONFIG_OPEN_JDK_JRE '{memory_calculator: {stack_threads: 25}}' -``` - -### B.2 Container Configuration - -```bash -# Use Tomcat 10.1.x -cf set-env my-app JBP_CONFIG_TOMCAT '{tomcat: {version: 10.1.+}}' - -# Configure Groovy version -cf set-env my-app JBP_CONFIG_GROOVY '{groovy: {version: 4.0.+}}' - -# Java Main classpath -cf set-env my-app JBP_CONFIG_JAVA_MAIN '{arguments: "--server.port=9090"}' -``` - -### B.3 Framework Configuration - -```bash -# Enable New Relic -cf set-env my-app JBP_CONFIG_NEW_RELIC_AGENT '{enabled: true}' - -# Enable Debug (JDWP) -cf set-env my-app JBP_CONFIG_DEBUG '{enabled: true}' - -# Configure JMX -cf set-env my-app JBP_CONFIG_JMX '{enabled: true, port: 5000}' - -# Disable Spring Auto-Reconfiguration -cf set-env my-app JBP_CONFIG_SPRING_AUTO_RECONFIGURATION '{enabled: false}' -``` - ---- - -## Appendix C: Related Documentation - -- **GAP_ANALYSIS.md**: Original gap analysis (Session 22) -- **FEATURE_COMPARISON.md**: Detailed feature comparison -- **ruby_vs_go_buildpack_comparison.md**: Dependency installation comparison -- **MIGRATION_STATUS.md**: Migration progress tracking -- **FRAMEWORK_STATUS.md**: Framework implementation status -- **TESTING_JRE_PROVIDERS.md**: JRE testing guide - ---- - -**Document Version**: 1.1 -**Last Updated**: December 16, 2025 -**Next Review**: After final 3 framework implementations diff --git a/docs/DEVELOPING.md b/docs/DEVELOPING.md index 641d2b253..3400ea3ab 100644 --- a/docs/DEVELOPING.md +++ b/docs/DEVELOPING.md @@ -102,30 +102,37 @@ This creates executables in the `bin/` directory: ``` java-buildpack/ -├── bin/ # Compiled binaries (generated) -│ ├── supply # Supply phase executable -│ └── finalize # Finalize phase executable -├── src/java/ # Go source code -│ ├── containers/ # Container implementations (8 types) -│ ├── frameworks/ # Framework implementations (38 types) -│ ├── jres/ # JRE implementations (7 providers) -│ ├── supply/cli/ # Supply phase entrypoint -│ ├── finalize/cli/ # Finalize phase entrypoint -│ └── integration/ # Integration tests -├── config/ # YAML configuration files -│ ├── components.yml # Component registry -│ ├── open_jdk_jre.yml # Example: OpenJDK configuration -│ └── ... # Component-specific configs -├── scripts/ # Build and test scripts -│ ├── build.sh # Build binaries -│ ├── unit.sh # Run unit tests -│ ├── integration.sh # Run integration tests -│ └── package.sh # Package buildpack for deployment -├── vendor/ # Vendored Go dependencies -├── go.mod # Go module definition -├── go.sum # Dependency checksums -├── manifest.yml # Buildpack manifest -└── VERSION # Version number +├── bin/ # Phase scripts +│ ├── compile # Compile phase script +│ ├── detect # Detect phase script +│ ├── supply # Supply phase script +│ ├── release # Release phase script +│ └── finalize # Finalize phase script +├── ci/ # ci scripts +├── docs/ # Detailes docs about frameworks, development and testing +├── src/java/ # Go source code +| ├── common/ # Common code and libbuildpack context +│ ├── containers/ # Container implementations (8 types) +│ ├── frameworks/ # Framework implementations (38 types) +│ ├── hooks/ # libbuildpack hooks +│ ├── jres/ # JRE implementations (7 providers) +│ ├── supply/cli/ # Supply phase entrypoint +│ ├── finalize/cli/ # Finalize phase entrypoint +│ ├── resources/ # Resource configuration files +│ └── integration/ # Integration tests +├── scripts/ # Build and test scripts +│ ├── build.sh # Build binaries +│ ├── unit.sh # Run unit tests +│ ├── integration.sh # Run integration tests +│ └── package.sh # Package buildpack for deployment +├── vendor/ # Vendored Go dependencies +├── ARCHITECTURE.md # Detailed architecture guide +├── RUBY_VS_GO_BUILDPACK_COMPARISON.md # Ruby vs Go buildpack implementations comparison info +├── CONTRIBUTING.md # Contribution guide +├── go.mod # Go module definition +├── go.sum # Dependency checksums +├── manifest.yml # Buildpack manifest +└── VERSION # Version number Key Go Packages: - containers/ - Application container implementations (Tomcat, Spring Boot, etc.) @@ -310,6 +317,8 @@ cd src/java ginkgo watch -r frameworks/ ``` +For detailed guidelines about setting up and running tests you can also check [Testing Guide](docs/TESTING.md) + ## Development Workflow ### Typical Development Cycle diff --git a/docs/custom-jre-usage.md b/docs/custom-jre-usage.md index cd1bdd675..bb5d79ac3 100644 --- a/docs/custom-jre-usage.md +++ b/docs/custom-jre-usage.md @@ -178,7 +178,7 @@ applications: - my-custom-java-buildpack env: BP_JAVA_VERSION: 17 - JBP_CONFIG_COMPONENTS: '{"jres": ["OracleJRE"]}' + JBP_CONFIG_ORACLE_JRE: '{ jre: { version: 17.+ }}' ``` **Option B: Specify buildpack on command line** @@ -191,7 +191,7 @@ cf push my-app -b my-custom-java-buildpack cf set-env my-app BP_JAVA_VERSION 17 # Select JRE vendor (if multiple JREs available) -cf set-env my-app JBP_CONFIG_COMPONENTS '{"jres": ["GraalVMJRE"]}' +cf set-env my-app JBP_CONFIG_GRAAL_VM_JRE '{ jre: { version: 21.+ }}' # Restage to apply changes cf restage my-app @@ -235,7 +235,7 @@ dependencies: # Push application with Oracle JRE 17 cf push my-app -b my-custom-java-buildpack cf set-env my-app BP_JAVA_VERSION 17 -cf set-env my-app JBP_CONFIG_COMPONENTS '{"jres": ["OracleJRE"]}' +cf set-env my-app JBP_CONFIG_ORACLE_JRE '{ jre: { version: 17.+ }}' cf restage my-app ``` diff --git a/manifest.yml b/manifest.yml index 1aecc2d5c..084ba8e31 100644 --- a/manifest.yml +++ b/manifest.yml @@ -68,6 +68,10 @@ default_versions: version: 6.x - name: jrebel version: 2025.x +- name: java-memory-assistant + version: 0.x +- name: java-memory-assistant-cleanup + version: 0.x - name: your-kit-profiler version: 2025.x - name: jprofiler-profiler @@ -159,6 +163,12 @@ url_to_dependency_map: - match: jrebel-(\d+\.\d+\.\d+) name: jrebel version: $1 +- match: java-memory-assistant-(\d+\.\d+\.\d+) + name: java-memory-assistant + version: $1 +- match: java-memory-assistant-cleanup-(\d+\.\d+\.\d+) + name: java-memory-assistant-cleanup + version: $1 - match: YourKit-JavaProfiler-(\d+\.\d+) name: your-kit-profiler version: $1 @@ -522,6 +532,22 @@ dependencies: cf_stacks: - cflinuxfs4 +# Java Memory Assistant +- name: java-memory-assistant + version: 0.5.0 + uri: https://github.com/SAP/java-memory-assistant/releases/download/0.5.0/java-memory-assistant-0.5.0.jar + sha256: 9c5ffb4bdeec5ed6b4f1d734469500754a857d1452c3d253d89e2315addb04c5 + cf_stacks: + - cflinuxfs4 + +# Java Memory Assistant Clean-up +- name: java-memory-assistant-cleanup + version: 0.1.0 + uri: https://github.com/SAP/java-memory-assistant-tools/releases/download/0.1.0/cleanup-linux-amd64-0.1.0.zip + sha256: f6a0348f70981ad3b7bc84d53f5964e5f60070ef75ca1ddac9184a4ab1639f75 + cf_stacks: + - cflinuxfs4 + # YourKit Java Profiler - name: your-kit-profiler version: 2025.9