Skip to content

Conversation

@simbo1905
Copy link
Owner

Summary

Resolves #16 by converting the JSON Test Suite from a failing unit test that blocks builds into a comprehensive standalone reporting tool.

Key Changes

  • Migrated JsonTestSuiteSummary from test to main using git mv, converted to executable tool with main() method
  • Added Maven wrapper and exec plugin configuration for easy execution
  • Disabled blocking unit test with @Disabled annotation - builds no longer fail
  • Implemented robust encoding detection via RobustCharDecoder to handle malformed UTF-8, BOM detection, UTF-16/UTF-32, and permissive byte-to-char conversion
  • Comprehensive API testing - now tests both Json.parse(String) and Json.parse(char[]) APIs
  • Added security documentation warning about undocumented StackOverflowError vulnerabilities in upstream API

Test Results

  • 318 files analyzed (0 skipped due to robust encoding)
  • 99.3% overall conformance maintained
  • 2 duplicate key failures are implementation choice, not bugs
  • 26 files successfully processed via robust encoding detection

Usage

# Build and download test suite
./mvnw clean compile generate-test-resources -pl json-compatibility-suite

# Run human-readable report
./mvnw exec:java -pl json-compatibility-suite

# Run JSON output
./mvnw exec:java -pl json-compatibility-suite -Dexec.args="--json"

The tool now provides comprehensive JSON Test Suite analysis without blocking builds, exactly addressing the issue raised.

…oding

- Convert JsonTestSuiteSummary from blocking unit test to executable tool
- Add Maven wrapper and exec plugin configuration
- Disable unit test with @disabled to prevent build failures
- Implement RobustCharDecoder for comprehensive encoding detection (BOM, UTF-16/32, permissive UTF-8)
- Add security documentation warning about undocumented StackOverflowError vulnerabilities
- Achieve 0 files skipped, 99.3% conformance across 318 test files
- Support both human-readable and JSON output formats

Resolves #16
@simbo1905 simbo1905 merged commit f67129e into main Jul 28, 2025
2 checks passed
@simbo1905 simbo1905 deleted the json-test-suite branch July 28, 2025 06:37
simbo1905 pushed a commit that referenced this pull request Aug 22, 2025
…ng tool (#17)

* compat test but got failures

* Refactor JSON Test Suite to standalone reporting tool with robust encoding

- Convert JsonTestSuiteSummary from blocking unit test to executable tool
- Add Maven wrapper and exec plugin configuration
- Disable unit test with @disabled to prevent build failures
- Implement RobustCharDecoder for comprehensive encoding detection (BOM, UTF-16/32, permissive UTF-8)
- Add security documentation warning about undocumented StackOverflowError vulnerabilities
- Achieve 0 files skipped, 99.3% conformance across 318 test files
- Support both human-readable and JSON output formats

Resolves #16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JSONTestSuite has 2 failures out of 318 tests - is this a back port regression?

2 participants