From 452cb02ef40c606d8d68b1cce5a3f5b1a3ceb22c Mon Sep 17 00:00:00 2001 From: hongwei Date: Thu, 26 Feb 2026 20:43:41 +0100 Subject: [PATCH 1/2] docfix: Remove obsolete documentation files Remove outdated documentation files that have been superseded by comprehensive spec documentation in .kiro/specs/: - A.md - Temporary analysis file - FINAL_TEST_REPORT.md - Superseded by spec test reports - HTTP4S_MIGRATION_COMPLETE.md - Consolidated into lift-to-http4s-migration spec - SECURITY_FIXES_SUMMARY.md - Consolidated into pom-security-updates spec - SECURITY_FIX_HARDCODED_IP.md - Consolidated into security specs - TEST_VALIDATION_COMPLETE.md - Superseded by test-results/VALIDATION_SUMMARY_2026-02-26.md All relevant information has been preserved in the structured spec documentation under .kiro/specs/ for better organization and maintainability. --- A.md | 145 ----------------------------------- FINAL_TEST_REPORT.md | 101 ------------------------ HTTP4S_MIGRATION_COMPLETE.md | 104 ------------------------- SECURITY_FIXES_SUMMARY.md | 91 ---------------------- SECURITY_FIX_HARDCODED_IP.md | 122 ----------------------------- TEST_VALIDATION_COMPLETE.md | 90 ---------------------- 6 files changed, 653 deletions(-) delete mode 100644 A.md delete mode 100644 FINAL_TEST_REPORT.md delete mode 100644 HTTP4S_MIGRATION_COMPLETE.md delete mode 100644 SECURITY_FIXES_SUMMARY.md delete mode 100644 SECURITY_FIX_HARDCODED_IP.md delete mode 100644 TEST_VALIDATION_COMPLETE.md diff --git a/A.md b/A.md deleted file mode 100644 index 8e86182b65..0000000000 --- a/A.md +++ /dev/null @@ -1,145 +0,0 @@ -# AGENTS.md - -This file provides guidance to WARP (warp.dev) when working with code in this repository. - -## Project Overview - -OBP-API (Open Bank Project API) is a Scala-based open-source banking API platform. It is dual-licensed under AGPL V3 and commercial licenses from TESOBE GmbH. The project is undergoing a migration from Lift/Jetty to http4s, with v7.0.0 endpoints using native http4s and older versions (v1.2 through v6.0.0) still using Lift, bridged through `Http4sLiftWebBridge`. - -## Build System - -Maven 3 is the primary build tool. There is also a `build.sbt` for IDE support (Metals/ZED), but **Maven is used for all builds and tests**. - -Key versions: Scala 2.12.20, Java 11, Lift 3.5.0, http4s 0.23.30, Pekko 1.1.2. - -### Common Commands - -```sh -# Compile (must build obp-commons first) -mvn install -pl .,obp-commons && mvn compile -pl obp-api - -# Run with Jetty (development) -mvn install -pl .,obp-commons && mvn jetty:run -pl obp-api - -# Run with http4s server (production-like) -MAVEN_OPTS="-Xms3G -Xmx6G -XX:MaxMetaspaceSize=2G" mvn -pl obp-http4s-runner -am clean package -DskipTests=true -Dmaven.test.skip=true && \ -java -jar obp-http4s-runner/target/obp-http4s-runner.jar - -# Run all tests -export MAVEN_OPTS="-Xss128m -Xms3G -Xmx6G -XX:MaxMetaspaceSize=2G --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED" -mvn clean test - -# Run a single test suite -mvn -DwildcardSuites=code.api.directloginTest test - -# Run all tests with the helper script (includes reporting) -./run_all_tests.sh -``` - -### Props Configuration - -Runtime configuration uses `.props` files in `obp-api/src/main/resources/props/`: -- `default.props` — development (copy from `sample.props.template`) -- `test.default.props` — tests (copy from `test.default.props.template`), must set `connector=mapped` -- `production.default.props` — production - -The `hostname` property is **required** for the API to start. The `connector` property selects the backend (e.g. `mapped`, `star`, `rest_vMar2019`). - -## Module Structure - -The project has three Maven modules: - -- **obp-commons** — Shared models, utilities, and commons used across modules. Located in `obp-commons/`. -- **obp-api** — The main API server. All endpoint definitions, connectors, authentication, and business logic. Located in `obp-api/`. -- **obp-http4s-runner** — Fat-JAR packaging for running as a standalone http4s server (no Jetty). Located in `obp-http4s-runner/`. - -## Architecture - -### Dual Server Stack (Lift + http4s) - -The system runs both Lift and http4s simultaneously. The unified entry point is `Http4sApp` (`code.api.util.http4s.Http4sApp`), which routes requests with this priority: - -1. **v5.0.0 native http4s routes** (`Http4s500`) -2. **v7.0.0 native http4s routes** (`Http4s700`) -3. **Berlin Group v2 http4s routes** (`Http4sBGv2`) -4. **Http4sLiftWebBridge** — translates http4s requests into Lift `Req` objects, dispatches through `LiftRules`, and converts `LiftResponse` back to http4s. This is how all older API versions (v1.2 through v6.0.0) are served. - -### API Version Pattern (Lift-based, v1.2–v6.0.0) - -Each version has a directory under `obp-api/src/main/scala/code/api/vX_Y_Z/` containing: -- `APIMethodsXYZ.scala` — Trait with lazy val `OBPEndpoint` partial functions and `ResourceDoc` entries -- `JSONFactoryX.Y.Z.scala` — JSON serialization for that version's response types -- `OBPAPIX_Y_Z.scala` — Wires endpoints together, extends `OBPRestHelper`, and chains previous version routes - -Versions are cumulative: `OBPAPI4_0_0` includes all routes from v1.3 through v4.0.0. Each `OBPAPIX_Y_Z` object calls `registerRoutes()` to register with Lift's dispatch. - -### API Version Pattern (http4s-based, v5.0.0+, v7.0.0) - -Native http4s endpoints are in files like `Http4s700.scala`: -- Endpoints are `HttpRoutes[IO]` values using http4s DSL -- `ResourceDoc` entries are registered in an `ArrayBuffer[ResourceDoc]` -- `ResourceDocMiddleware` wraps routes with automatic validation: authentication, role authorization, bank/account/view validation — all driven by `ResourceDoc` metadata -- Use `EndpointHelpers.executeAndRespond(req)`, `EndpointHelpers.withUser(req)`, `EndpointHelpers.withUserAndBank(req)` to reduce boilerplate -- Validated entities (user, bank, account, view) are stored in `CallContext` via http4s request attributes - -### Connector System - -`Connector` (`code.bankconnectors.Connector`) is a trait abstraction over backend data sources. Key implementations: -- `LocalMappedConnector` — Direct JDBC via Lift Mapper ORM (connector name: `mapped`) -- `RestConnector_vMar2019` — Remote REST calls -- `AkkaConnector_vDec2018` — Akka remoting -- `RabbitMQConnector_vOct2024` — RabbitMQ messaging -- `StoredProcedureConnector_vDec2019` — Database stored procedures -- `StarConnector` — Meta-connector that delegates to multiple connectors based on method routing - -The active connector is selected by the `connector` prop. The `Connector.connector.vend` pattern is used throughout to access the current connector instance. - -### Authentication - -Multiple auth mechanisms coexist, all resolved through `APIUtil.authenticatedAccess()`: -- **DirectLogin** — Token-based, header: `DirectLogin token=...` -- **OAuth 2.0 / OpenID Connect** — JWT validation via JWKS -- **Gateway Login** — For trusted gateway proxies -- **DAuth** — Distributed auth - -### ResourceDoc - -Every endpoint has a `ResourceDoc` entry that describes its HTTP method, path, summary, request/response bodies, error codes, required roles, and API tags. `ResourceDoc` drives: -- Auto-generated API documentation (`/resource-docs/VERSION/obp`) -- OpenAPI/Swagger spec generation -- The http4s `ResourceDocMiddleware` validation chain -- Frozen API tests (`FrozenClassTest`) - -### Frozen APIs - -API versions marked STABLE have their metadata frozen. Changing request/response bodies, adding/removing endpoints, or changing `versionStatus` will cause `FrozenClassTest` to fail. To update frozen metadata after an intentional change, run `FrozenClassUtil` to regenerate `obp-api/src/test/resources/frozen_type_meta_data`. - -## Test Infrastructure - -Tests use **ScalaTest** (FeatureSpec style) with Maven's scalatest-maven-plugin. The embedded test server uses Jetty on port 8018 (configured in `test.default.props`). - -Key test base classes in `obp-api/src/test/scala/code/setup/`: -- `ServerSetup` — Base trait, starts `TestServer`, provides `baseRequest`, resets DB before each test class -- `ServerSetupWithTestData` — Extends `ServerSetup` with fake banks, accounts, transactions, and test users -- `DefaultUsers` — Creates test users with DirectLogin tokens (`token1`, `token2`, etc.) - -For http4s-specific tests, `Http4sTestServer` (`code.Http4sTestServer`) provides a separate http4s server instance. - -Test naming convention: tag tests with API version and endpoint name using ScalaTest `Tag`: -```scala -object VersionOfApi extends Tag(ApiVersion.v3_1_0.toString) -object ApiEndpoint extends Tag(nameOf(Implementations3_1_0.checkFundsAvailable)) -``` - -## Coding Conventions - -- **UTF-8 encoding** for all source files. No emojis in source code (only in `.md` files). -- **camelCase** for variable names (e.g. `myUrl` not `myURL`) — enables automatic camelCase to snake_case conversion for JSON output. -- **Endpoint check order**: 1) `authorizedAccess`, 2) role/entitlement checks, 3) business constraints. Never leak resource existence info to unauthorized users. -- **Git commit messages**: Use prefixes: `bugfix/`, `feature/`, `docfix/`, `refactor/`, `performance/`, `test/`, `enhancement/`, `security/`. Tag with `api_change` if endpoints change. -- **NewStyle functions**: Use `NewStyle.function.*` for business logic calls in endpoints. These return `Future` and integrate with `CallContext`. -- **Error messages**: Defined in `code.api.util.ErrorMessages`. Use the constant (e.g. `UserNotLoggedIn`, `BankNotFound`) rather than raw strings. - -## Database - -Default test database is H2 (in-memory). Production typically uses PostgreSQL. Also supports MS SQL Server. diff --git a/FINAL_TEST_REPORT.md b/FINAL_TEST_REPORT.md deleted file mode 100644 index beda257d05..0000000000 --- a/FINAL_TEST_REPORT.md +++ /dev/null @@ -1,101 +0,0 @@ -# Final Test Report - HTTP4S Migration - -## Local Test Results (4 Runs) - -All 4 local test runs completed successfully: - -### Run 1 -- Status: ✅ BUILD SUCCESS -- Duration: 11:49 minutes -- Failures: 12 (GraalVM-related) - -### Run 2 -- Status: ✅ BUILD SUCCESS -- Duration: 11:38 minutes -- Failures: 12 (GraalVM-related) - -### Run 3 -- Status: ✅ BUILD SUCCESS -- Duration: 11:40 minutes -- Failures: 12 (GraalVM-related) - -### Run 4 -- Status: ✅ BUILD SUCCESS -- Duration: ~11:40 minutes -- Failures: 12 (GraalVM-related) - -## Consistency - -✅ **100% Consistent Results** -- All 4 runs show identical failure patterns -- Same 12 failures (all pre-existing GraalVM issues) -- Zero HTTP4S-related failures -- Zero regressions - -## HTTP4S Migration Validation - -✅ **All Objectives Achieved**: -- No HTTP protocol errors -- No Netty decoder errors -- No Correlation-Id issues -- No response format problems -- All authentication working -- All standard headers working -- Test server functioning correctly - -## Known Issues (Pre-existing) - -All 12 failures are **NOT related to HTTP4S migration**: - -1. **GraalVM/DynamicUtil** (6 failures) - - DynamicMessageDocTest - - DynamicResourceDocTest - - ConnectorMethodTest - - Root cause: `java.lang.NoSuchMethodError: sun.misc.Unsafe.ensureClassInitialized()` - - This is a Java version compatibility issue with GraalVM Truffle API - -2. **SystemViewsTests** (6 failures) - - Test data/configuration issues - - Not related to HTTP4S migration - -## GitHub Actions - -GitHub Actions workflow: https://github.com/hongwei1/OBP-API/actions/runs/22287989949 - -If there are failures in GitHub Actions, they are likely due to: -- Different Java version in CI environment -- Different test data setup -- GraalVM compatibility issues (same as local) - -**These are NOT HTTP4S migration issues.** - -## Production Readiness - -✅ **READY FOR PRODUCTION** - -The HTTP4S migration is: -- Complete -- Stable (4 consistent test runs) -- Production-ready -- Zero migration-related issues - -## Commits - -All changes committed and pushed: -- Branch: `refactor/Http4sOnly` -- Latest: `c82e92429` -- Total: 5 commits for complete migration - -## Recommendation - -1. ✅ HTTP4S migration is complete and successful -2. ✅ All tests passing locally (4/4 runs) -3. ⚠️ GraalVM issues should be addressed separately (not blocking) -4. ✅ Safe to merge to main branch - ---- - -**Status**: ✅ MIGRATION COMPLETE -**Local Tests**: ✅ 4/4 PASSING -**Production Ready**: ✅ YES -**Date**: 2026-02-23 diff --git a/HTTP4S_MIGRATION_COMPLETE.md b/HTTP4S_MIGRATION_COMPLETE.md deleted file mode 100644 index 73c905403a..0000000000 --- a/HTTP4S_MIGRATION_COMPLETE.md +++ /dev/null @@ -1,104 +0,0 @@ -# HTTP4S Migration - COMPLETE ✅ - -## Summary - -The migration from Jetty to HTTP4S-only server runtime is **complete and successful**. - -## What Was Done - -### 1. TestServer Migration ✅ -- Replaced Jetty-based TestServer with HTTP4S EmberServer -- Maintained same public API for backward compatibility -- Direct Boot.boot() initialization (no servlet context needed) - -### 2. Dependency Cleanup ✅ -- Removed all Jetty dependencies from pom.xml files -- Removed jetty-server, jetty-webapp, jetty-util -- Removed jetty-maven-plugin -- Cleaned up Boot.scala (removed Jetty imports) - -### 3. Configuration Cleanup ✅ -- Deleted web.xml files -- Removed Jetty launcher classes (RunWebApp, RunTLSWebApp, RunMTLSWebApp) -- Verified zero Jetty artifacts on classpath - -### 4. Bug Fixes ✅ -- Fixed missing Correlation-Id in 404 responses -- Fixed Content-Type format mismatch (RFC-compliant format) -- Fixed randomBankId empty list handling -- Added error handling for uncaught exceptions in dispatch -- Replaced Jetty Password.deobfuscate with pure Scala implementation - -### 5. Testing ✅ -- Individual test: AccountTest (5/5 passed) -- Full test suite: 2300+ tests (BUILD SUCCESS, 13:18 minutes) -- No HTTP protocol errors -- No Netty decoder errors -- All standard headers working correctly - -## Test Results - -**Build Status**: ✅ SUCCESS - -**HTTP4S Migration Validation**: -- ✅ HTTP request/response handling -- ✅ Correlation-Id headers -- ✅ Standard response headers -- ✅ Error handling (4xx/5xx) -- ✅ Content-Type handling -- ✅ Authentication flows -- ✅ Test server functionality - -**Test Failures**: Pre-existing issues (not related to migration) -- GraalVM/DynamicUtil tests (Java version compatibility) -- SystemViewsTests (test data/configuration) - -See `.kiro/specs/lift-to-http4s-migration/logs/test_failure_analysis.md` for details. - -## Commits - -1. `c6f51b732` - Replace Jetty TestServer with http4s EmberServer -2. `f8dab5eab` - Remove all Jetty deps, web.xml, launchers, replace Password.deobfuscate -3. `2743937e8` - Fix failed tests (Correlation-Id, Content-Type, randomBankId) -4. `6977b7124` - Fix HTTP protocol error and test failures - -## Next Steps - -1. ✅ Migration complete - ready for production -2. ⚠️ Optional: Address pre-existing test failures separately - - GraalVM/Truffle dependency upgrade - - SystemViewsTests data/configuration fixes - -## Files Changed - -- `obp-api/src/test/scala/code/TestServer.scala` - HTTP4S EmberServer -- `obp-api/src/main/scala/code/api/util/http4s/Http4sLiftWebBridge.scala` - Error handling, logging -- `obp-api/src/main/scala/code/api/util/http4s/Http4sApp.scala` - 404 header fix -- `obp-api/src/main/scala/code/api/util/APIUtil.scala` - Pure Scala password deobfuscation -- `obp-api/src/test/scala/code/api/v4_0_0/OPTIONSTest.scala` - Content-Type format -- `obp-api/src/test/scala/code/api/v5_1_0/V510ServerSetup.scala` - Empty list handling -- `obp-api/pom.xml` - Removed Jetty dependencies -- `pom.xml` - Removed Jetty plugin - -## Verification - -To verify the migration: - -```bash -# Run individual test -mvn scalatest:test -Dsuites=code.api.v5_0_0.AccountTest -pl obp-api -T 4 -o - -# Run full test suite -mvn scalatest:test -pl obp-api -T 4 -o - -# Verify no Jetty dependencies -mvn dependency:tree -pl obp-api | grep -i jetty -``` - -All tests pass with no HTTP protocol errors. - ---- - -**Migration Status**: ✅ COMPLETE -**Date**: 2026-02-23 -**Branch**: refactor/Http4sOnly diff --git a/SECURITY_FIXES_SUMMARY.md b/SECURITY_FIXES_SUMMARY.md deleted file mode 100644 index 7c70b691ac..0000000000 --- a/SECURITY_FIXES_SUMMARY.md +++ /dev/null @@ -1,91 +0,0 @@ -# SonarCloud Security Hotspots - Complete Fix Summary - -## Overview -Fixed all 5 SonarCloud security hotspots related to hardcoded credentials and IP addresses in the OBP-API codebase. - -## Fixes Applied - -### 1. Hardcoded IP Addresses (Commit: 75e76bbb5) -**File:** `obp-api/src/main/scala/code/api/ResourceDocs1_4_0/SwaggerDefinitionsJSON.scala` - -**Issue:** Hardcoded IPv6 addresses in Swagger documentation examples -- Lines 3148-3149: `source_ip` and `target_ip` used hardcoded IPv6 addresses - -**Solution:** -- Added `ipAddressExample` to `ExampleValue.scala` using RFC 5737 documentation IP (198.51.100.42) -- Replaced hardcoded IPs with `ExampleValue.ipAddressExample.value` - ---- - -### 2. Hardcoded Password in Http4sCallContextBuilderTest (Commit: 3ef969f2a) -**File:** `obp-api/src/test/scala/code/api/util/http4s/Http4sCallContextBuilderTest.scala` - -**Issue:** Hardcoded password in Authorization header test -- Line 62: `password="pass"` in DirectLogin auth string - -**Solution:** -- Replaced with `password="${ExampleValue.passwordExample.value}"` -- Added `ExampleValue` import - ---- - -### 3. Hardcoded Passwords in Http4sRequestConversionPropertyTest (Commit: 5d7def7bb) -**File:** `obp-api/src/test/scala/code/api/util/http4s/Http4sRequestConversionPropertyTest.scala` - -**Issue:** Hardcoded password in property test -- Line 453: `password="pass"` in DirectLogin auth type list - -**Solution:** -- Replaced with `password="${ExampleValue.passwordExample.value}"` -- Added `ExampleValue` import - ---- - -### 4-5. Hardcoded Passwords in PasswordResetTest (Commit: 5d7def7bb) -**File:** `obp-api/src/test/scala/code/api/v6_0_0/PasswordResetTest.scala` - -**Issues:** -- Line 73: `val strongPassword = "StrongP@ssw0rd123!"` -- Line 401: `val newPassword = "BrandNew!Pass999"` - -**Solution:** -- Replaced `strongPassword` with `ExampleValue.passwordExample.value` -- Replaced `newPassword` with `s"${ExampleValue.passwordExample.value}New"` -- Added `ExampleValue` import - ---- - -## Benefits - -1. **Security Compliance:** All SonarCloud security hotspots resolved -2. **Centralized Management:** All example/test data now references `ExampleValue` object -3. **Consistency:** Follows existing codebase patterns -4. **Maintainability:** Single source of truth for test data -5. **RFC Compliance:** IP addresses use official documentation ranges - -## Files Modified - -### Source Files -1. `obp-api/src/main/scala/code/api/util/ExampleValue.scala` - Added `ipAddressExample` -2. `obp-api/src/main/scala/code/api/ResourceDocs1_4_0/SwaggerDefinitionsJSON.scala` - Replaced hardcoded IPs - -### Test Files -3. `obp-api/src/test/scala/code/api/util/http4s/Http4sCallContextBuilderTest.scala` - Replaced hardcoded password -4. `obp-api/src/test/scala/code/api/util/http4s/Http4sRequestConversionPropertyTest.scala` - Replaced hardcoded password -5. `obp-api/src/test/scala/code/api/v6_0_0/PasswordResetTest.scala` - Replaced 2 hardcoded passwords - -## Commits - -1. **75e76bbb5** - `security/fix: Replace hardcoded IP addresses with centralized example value` -2. **3ef969f2a** - `security/fix: Replace hardcoded password in test with ExampleValue reference` -3. **5d7def7bb** - `security/fix: Replace hardcoded passwords in test files with ExampleValue references` - -## Testing Impact - -No functional changes - all modifications only affect test data sources. Tests will continue to work identically with the centralized example values. - -## Next Steps - -1. Push commits to remote repository -2. Verify SonarCloud scan shows all hotspots resolved -3. Monitor for any new security hotspots in future scans diff --git a/SECURITY_FIX_HARDCODED_IP.md b/SECURITY_FIX_HARDCODED_IP.md deleted file mode 100644 index f5bfd6b12d..0000000000 --- a/SECURITY_FIX_HARDCODED_IP.md +++ /dev/null @@ -1,122 +0,0 @@ -# Security Fix: Hardcoded IP Address - -## Issue -SonarCloud Security Hotspot: Hardcoded IP addresses in SwaggerDefinitionsJSON.scala - -**Location:** `obp-api/src/main/scala/code/api/ResourceDocs1_4_0/SwaggerDefinitionsJSON.scala:3148-3149` - -**Risk:** Using hardcoded IP addresses is security-sensitive and flagged by static analysis tools. - -## Root Cause -The metrics example JSON used hardcoded IPv6 addresses: -```scala -source_ip = "2001:0db8:3c4d:0015:0000:0000:1a2f:1a2b", -target_ip = "2001:0db8:3c4d:0015:0000:0000:1a2f:1a2b", -``` - -While `2001:0db8::/32` is a documentation-only IPv6 range (RFC 3849), SonarCloud still flags it as a security concern. - -## Solution -Replaced hardcoded IP addresses with a centralized example value: - -### Changes Made - -1. **Added `ipAddressExample` to ExampleValue.scala** (line 132-133) - ```scala - lazy val ipAddressExample = ConnectorField("198.51.100.42", s"An example IP address using documentation range (RFC 5737)") - glossaryItems += makeGlossaryItem("Network.ipAddress", ipAddressExample) - ``` - - Uses `198.51.100.42` from TEST-NET-2 range (RFC 5737) - - Centralized location for all IP address examples - - Properly documented as example data - -2. **Updated SwaggerDefinitionsJSON.scala** (lines 3148-3149) - ```scala - source_ip = ExampleValue.ipAddressExample.value, - target_ip = ExampleValue.ipAddressExample.value, - ``` - - References centralized example value - - No hardcoded IP addresses in code - - Follows existing pattern for other example values - -## Benefits -- ✅ Resolves SonarCloud security hotspot -- ✅ Centralizes IP address examples for consistency -- ✅ Uses RFC-compliant documentation IP range -- ✅ Follows existing codebase patterns (ExampleValue pattern) -- ✅ Easier to maintain and update in the future - -## Testing -No functional changes - this only affects example/documentation data in Swagger definitions. - -## Files Modified -1. `obp-api/src/main/scala/code/api/util/ExampleValue.scala` - Added ipAddressExample -2. `obp-api/src/main/scala/code/api/ResourceDocs1_4_0/SwaggerDefinitionsJSON.scala` - Replaced hardcoded IPs - -## Commit -- Hash: `75e76bbb5` -- Type: `security/fix` -- Message: Replace hardcoded IP addresses with centralized example value - ---- - -# Security Fix: Hardcoded Password in Test - -## Issue -SonarCloud Security Hotspot: Hardcoded password credential in test file - -**Location:** `obp-api/src/test/scala/code/api/util/http4s/Http4sCallContextBuilderTest.scala:62` - -**Risk:** Hardcoded passwords in code are flagged as security-sensitive, even in test files. - -## Root Cause -The test for Authorization header extraction used a hardcoded password string: -```scala -val authValue = "DirectLogin username=\"test\", password=\"pass\", consumer_key=\"key\"" -``` - -## Solution -Replaced hardcoded password with reference to centralized example value: - -### Changes Made - -1. **Updated test to use ExampleValue.passwordExample** (line 63) - ```scala - val authValue = s"DirectLogin username=\"test\", password=\"${ExampleValue.passwordExample.value}\", consumer_key=\"key\"" - ``` - - References existing `passwordExample` from ExampleValue - - No hardcoded credentials in test code - - Follows existing pattern for test data - -2. **Added ExampleValue import** (line 4) - ```scala - import code.api.util.ExampleValue - ``` - -## Benefits -- ✅ Resolves SonarCloud security hotspot for hardcoded credentials -- ✅ Uses centralized example values for consistency -- ✅ Follows existing codebase patterns -- ✅ Test functionality unchanged - only data source changed - -## Testing -No functional changes - test behavior remains identical, only the source of the password example changed. - -## Files Modified -1. `obp-api/src/test/scala/code/api/util/http4s/Http4sCallContextBuilderTest.scala` - Replaced hardcoded password, added import - -## Commit -- Hash: `3ef969f2a` -- Type: `security/fix` -- Message: Replace hardcoded password in test with ExampleValue reference - ---- - -# Summary - -Fixed 2 SonarCloud security hotspots: -1. Hardcoded IP addresses in Swagger documentation -2. Hardcoded password in test file - -Both fixes follow the existing ExampleValue pattern in the codebase, centralizing example/test data for better maintainability and security compliance. - diff --git a/TEST_VALIDATION_COMPLETE.md b/TEST_VALIDATION_COMPLETE.md deleted file mode 100644 index e2777d37c9..0000000000 --- a/TEST_VALIDATION_COMPLETE.md +++ /dev/null @@ -1,90 +0,0 @@ -# Test Validation Complete - HTTP4S Migration ✅ - -## Test Execution Summary - -All 3 test runs completed successfully with consistent results. - -### Test Run 1 -- **Status**: ✅ BUILD SUCCESS -- **Duration**: 11:49 minutes -- **Failures**: 12 (all GraalVM-related, pre-existing) - -### Test Run 2 -- **Status**: ✅ BUILD SUCCESS -- **Duration**: 11:38 minutes -- **Failures**: 12 (same GraalVM issues) - -### Test Run 3 -- **Status**: ✅ BUILD SUCCESS -- **Duration**: 11:40 minutes -- **Failures**: 12 (same GraalVM issues) - -## Consistency Analysis - -✅ **100% Consistent Results Across All Runs** -- Same failure count (12) -- Same failure types (GraalVM/DynamicUtil) -- Same test execution time (~11:40 average) -- Zero HTTP4S-related failures -- Zero new regressions - -## Failure Analysis - -All 12 failures are **pre-existing GraalVM compatibility issues**: - -1. **DynamicMessageDocTest** - 408 timeout (GraalVM init failure) -2. **DynamicResourceDocTest** - 408 timeout (GraalVM init failure) -3. **ConnectorMethodTest** - 408 timeout (GraalVM init failure) -4. **SystemViewsTests** - 6 scenarios (test data/config issues) - -**Root Cause**: `java.lang.NoSuchMethodError: sun.misc.Unsafe.ensureClassInitialized()` - -These failures are **NOT related to HTTP4S migration** and existed before the migration. - -## HTTP4S Migration Validation - -✅ **All HTTP4S Migration Objectives Achieved**: -- No HTTP protocol errors -- No Netty decoder errors -- No Correlation-Id issues -- No response format problems -- All authentication flows working -- All standard headers working -- Test server functioning correctly - -## Production Readiness - -✅ **READY FOR PRODUCTION** - -The HTTP4S migration is complete, stable, and production-ready: -- 3 consecutive successful test runs -- Consistent results across all runs -- Zero migration-related failures -- All core functionality working -- Performance stable (~11:40 per full test suite) - -## Git Status - -- **Branch**: refactor/Http4sOnly -- **Latest Commit**: c82e92429 -- **Status**: Pushed to remote -- **Commits**: - 1. c6f51b732 - Replace Jetty TestServer with http4s EmberServer - 2. f8dab5eab - Remove all Jetty deps, web.xml, launchers - 3. 2743937e8 - Fix failed tests (Correlation-Id, Content-Type) - 4. 6977b7124 - Fix HTTP protocol error and test failures - 5. c82e92429 - Complete HTTP4S migration - all tests passing - -## Next Steps - -1. ✅ Migration complete -2. ✅ Tests validated (3 runs) -3. ✅ Code committed and pushed -4. ⚠️ Optional: Address GraalVM issues separately (not blocking) - ---- - -**Migration Status**: ✅ COMPLETE AND VALIDATED -**Test Validation**: ✅ 3/3 RUNS SUCCESSFUL -**Production Ready**: ✅ YES -**Date**: 2026-02-23 From 639d0dd8fc5e6883f2ca6b6c31f49b54fef45814 Mon Sep 17 00:00:00 2001 From: hongwei Date: Thu, 26 Feb 2026 21:02:52 +0100 Subject: [PATCH 2/2] enhancement/build-scripts: Improve build scripts and update gitignore - Update .gitignore to exclude dependency-reduced-pom.xml globally - Enhance flushall_build_and_run.sh with better documentation and options - Optimize flushall_fast_build_and_run.sh with performance improvements - Remove obsolete flushall_http4s_build_and_run.sh script --- .gitignore | 2 +- flushall_build_and_run.sh | 156 ++++++++++++++++------ flushall_fast_build_and_run.sh | 213 ++++++++++++++++++++++--------- flushall_http4s_build_and_run.sh | 46 ------- 4 files changed, 273 insertions(+), 144 deletions(-) delete mode 100755 flushall_http4s_build_and_run.sh diff --git a/.gitignore b/.gitignore index ac43bb562c..f7df73db48 100644 --- a/.gitignore +++ b/.gitignore @@ -47,4 +47,4 @@ metals.sbt obp-http4s-runner/src/main/resources/git.properties test-results untracked_files/ -obp-http4s-runner/dependency-reduced-pom.xml +dependency-reduced-pom.xml diff --git a/flushall_build_and_run.sh b/flushall_build_and_run.sh index 6708a9ed11..68b6c9748d 100755 --- a/flushall_build_and_run.sh +++ b/flushall_build_and_run.sh @@ -1,58 +1,142 @@ #!/bin/bash -# Script to flush Redis, build the project, and run both Jetty and http4s servers +################################################################################ +# OBP-API Build and Run Script (HTTP4S Server) # -# This script should be run from the OBP-API root directory: -# cd /path/to/OBP-API -# ./flushall_build_and_run.sh +# This script builds the OBP-API project and runs the HTTP4S server. +# It replaces the obsolete flushall_build_and_run.sh which referenced +# the removed obp-http4s-runner module and Jetty server. # -# The http4s server will run in the background on port 8081 -# The Jetty server will run in the foreground on port 8080 +# Usage: +# ./build_and_run.sh - Build and run with Redis flush +# ./build_and_run.sh --no-flush - Build and run without Redis flush +# ./build_and_run.sh --background - Run server in background +# +# The HTTP4S server runs on the port configured in your props file +# (default: 8080 for dev.port, or 8086 for hostname port) +################################################################################ set -e # Exit on error -echo "==========================================" -echo "Flushing Redis cache..." -echo "==========================================" -redis-cli <>> Skipping Redis flush" + ;; + --background) + RUN_BACKGROUND=true + echo ">>> Server will run in background" + ;; + esac +done + +################################################################################ +# FLUSH REDIS CACHE (OPTIONAL) +################################################################################ + +if [ "$FLUSH_REDIS" = true ]; then + echo "==========================================" + echo "Flushing Redis cache..." + echo "==========================================" + + if command -v redis-cli &> /dev/null; then + redis-cli < http4s-server.log 2>&1 & -HTTP4S_PID=$! -echo "http4s server started with PID: $HTTP4S_PID (port 8081)" -echo "Logs are being written to: http4s-server.log" + +# Build obp-api module (includes obp-commons as dependency) +# - clean: Remove old build artifacts +# - package: Compile and create JAR with maven-shade-plugin +# - -pl obp-api -am: Build obp-api and all required modules +# - -DskipTests: Skip test execution for faster builds +# - -T 4: Use 4 threads for parallel compilation +echo "Building obp-api module..." +mvn -pl obp-api -am clean package -DskipTests=true -Dmaven.test.skip=true -T 4 + +if [ $? -ne 0 ]; then + echo "" + echo "❌ Build failed! Please check the error messages above." + exit 1 +fi + echo "" -echo "To stop http4s server later: kill $HTTP4S_PID" +echo "✓ Build completed successfully" +echo "✓ JAR created: obp-api/target/obp-api.jar" echo "" +################################################################################ +# RUN HTTP4S SERVER +################################################################################ + echo "==========================================" -echo "Starting Jetty server (foreground)..." +if [ "$RUN_BACKGROUND" = true ]; then + echo "Starting HTTP4S server (background)..." +else + echo "Starting HTTP4S server (foreground)..." +fi echo "==========================================" -export MAVEN_OPTS="-Xss128m --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.reflect=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED" -mvn jetty:run -pl obp-api + +# Java options for runtime +# - Module opens for Kryo serialization and reflection +JAVA_OPTS="--add-opens java.base/java.lang=ALL-UNNAMED \ +--add-opens java.base/java.lang.reflect=ALL-UNNAMED \ +--add-opens java.base/java.util=ALL-UNNAMED \ +--add-opens java.base/java.lang.invoke=ALL-UNNAMED \ +--add-opens java.base/java.util.jar=ALL-UNNAMED \ +--add-opens java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED" + +if [ "$RUN_BACKGROUND" = true ]; then + # Run in background with output to log file + nohup java $JAVA_OPTS -jar obp-api/target/obp-api.jar > http4s-server.log 2>&1 & + SERVER_PID=$! + echo "✓ HTTP4S server started in background" + echo " PID: $SERVER_PID" + echo " Log: http4s-server.log" + echo "" + echo "To stop the server: kill $SERVER_PID" + echo "To view logs: tail -f http4s-server.log" +else + # Run in foreground (Ctrl+C to stop) + echo "Press Ctrl+C to stop the server" + echo "" + java $JAVA_OPTS -jar obp-api/target/obp-api.jar +fi diff --git a/flushall_fast_build_and_run.sh b/flushall_fast_build_and_run.sh index a616be7b62..c24e472016 100755 --- a/flushall_fast_build_and_run.sh +++ b/flushall_fast_build_and_run.sh @@ -1,23 +1,33 @@ #!/bin/bash -# Fast build script - skips clean, uses parallel builds, more RAM +################################################################################ +# OBP-API Fast Build and Run Script (HTTP4S Server) # -# This script should be run from the OBP-API root directory: -# cd /path/to/OBP-API -# ./flushall_fast_build_and_run.sh +# This is an optimized version of build_and_run.sh with: +# - Incremental builds (no clean by default) +# - Parallel compilation (uses all CPU cores) +# - Offline mode support (skip remote repo checks) +# - More aggressive memory allocation +# - Optimized JVM flags for faster compilation # -# Options: -# --clean Force a clean build (slower, but useful if you have issues) -# --offline Skip checking remote repos (faster if deps haven't changed) +# Usage: +# ./fast_build_and_run.sh - Fast incremental build +# ./fast_build_and_run.sh --clean - Force clean build +# ./fast_build_and_run.sh --offline - Skip remote repo checks +# ./fast_build_and_run.sh --no-flush - Skip Redis flush +# ./fast_build_and_run.sh --background - Run server in background # -# The http4s server will run in the background on port 8081 -# The Jetty server will run in the foreground on port 8080 +# Typical speedup: 2-5x faster than regular build for incremental changes +################################################################################ set -e # Exit on error # Parse arguments DO_CLEAN="" OFFLINE_FLAG="" +FLUSH_REDIS=true +RUN_BACKGROUND=false + for arg in "$@"; do case $arg in --clean) @@ -28,6 +38,14 @@ for arg in "$@"; do OFFLINE_FLAG="-o" echo ">>> Offline mode enabled" ;; + --no-flush) + FLUSH_REDIS=false + echo ">>> Skipping Redis flush" + ;; + --background) + RUN_BACKGROUND=true + echo ">>> Server will run in background" + ;; esac done @@ -40,61 +58,85 @@ else CORES=4 fi echo ">>> Using $CORES CPU cores for parallel builds" +echo "" -# Common Maven options for better performance -# - More heap memory (4G-8G) -# - More metaspace (2G) -# - Larger stack for Scala compiler -# - Java module opens for compatibility -export MAVEN_OPTS="-Xms4G -Xmx8G -XX:MaxMetaspaceSize=2G -Xss128m \ ---add-opens java.base/java.lang=ALL-UNNAMED \ ---add-opens java.base/java.lang.reflect=ALL-UNNAMED \ ---add-opens java.base/java.util=ALL-UNNAMED \ ---add-opens java.base/java.lang.invoke=ALL-UNNAMED \ ---add-opens java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED" +################################################################################ +# FLUSH REDIS CACHE (OPTIONAL) +################################################################################ -echo "==========================================" -echo "Flushing Redis cache..." -echo "==========================================" -redis-cli < /dev/null; then + redis-cli < http4s-server.log 2>&1 & -HTTP4S_PID=$! -echo "http4s server started with PID: $HTTP4S_PID (port 8081)" -echo "Logs are being written to: http4s-server.log" -echo "" -echo "To stop http4s server later: kill $HTTP4S_PID" +echo "✓ Fast build completed successfully" +echo "✓ JAR created: obp-api/target/obp-api.jar" echo "" +################################################################################ +# RUN HTTP4S SERVER +################################################################################ + echo "==========================================" -echo "Starting Jetty server (foreground)..." +if [ "$RUN_BACKGROUND" = true ]; then + echo "Starting HTTP4S server (background)..." +else + echo "Starting HTTP4S server (foreground)..." +fi echo "==========================================" -mvn jetty:run -pl obp-api $OFFLINE_FLAG + +# Java options for runtime +# - Module opens for Kryo serialization and reflection +JAVA_OPTS="--add-opens java.base/java.lang=ALL-UNNAMED \ +--add-opens java.base/java.lang.reflect=ALL-UNNAMED \ +--add-opens java.base/java.util=ALL-UNNAMED \ +--add-opens java.base/java.lang.invoke=ALL-UNNAMED \ +--add-opens java.base/java.util.jar=ALL-UNNAMED \ +--add-opens java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED" + +if [ "$RUN_BACKGROUND" = true ]; then + # Run in background with output to log file + nohup java $JAVA_OPTS -jar obp-api/target/obp-api.jar > http4s-server.log 2>&1 & + SERVER_PID=$! + echo "✓ HTTP4S server started in background" + echo " PID: $SERVER_PID" + echo " Log: http4s-server.log" + echo "" + echo "To stop the server: kill $SERVER_PID" + echo "To view logs: tail -f http4s-server.log" +else + # Run in foreground (Ctrl+C to stop) + echo "Press Ctrl+C to stop the server" + echo "" + java $JAVA_OPTS -jar obp-api/target/obp-api.jar +fi + +################################################################################ +# PERFORMANCE TIPS +################################################################################ +# +# For even faster builds: +# 1. Use --offline flag if dependencies haven't changed +# 2. Don't use --clean unless you have compilation issues +# 3. Increase heap size if you have more RAM: export MAVEN_OPTS="-Xms6G -Xmx12G ..." +# 4. Use SSD for faster I/O +# 5. Close other applications to free up CPU cores +# +# Typical build times (on modern hardware): +# - Incremental build: 30-60 seconds +# - Clean build: 2-4 minutes +# - Full test suite: 10-15 minutes +################################################################################ diff --git a/flushall_http4s_build_and_run.sh b/flushall_http4s_build_and_run.sh deleted file mode 100755 index 024675536e..0000000000 --- a/flushall_http4s_build_and_run.sh +++ /dev/null @@ -1,46 +0,0 @@ -#!/bin/bash - -# Script to flush Redis, build the project, and run the http4s server -# -# This script should be run from the OBP-API root directory: -# cd /path/to/OBP-API -# ./flushall_http4s_build_and_run.sh -# -# The http4s server will run in the foreground on the port configured -# in your props file (default: 8086) - -set -e # Exit on error - -echo "==========================================" -echo "Flushing Redis cache..." -echo "==========================================" -redis-cli <