# Node.js v16 to v20 Upgrade Plan

## Project Overview
- **Current Version**: Node.js v16 with npm v8
- **Target Version**: Node.js v20 with npm v10
- **Application Type**: Full-stack web application with Express.js backend and Next.js frontend
- **Key Technologies**: Express.js, Next.js, MongoDB (Mongoose), Sass, TypeScript

## Executive Summary

This upgrade involves a two-step process (v16 → v18 → v20) due to significant breaking changes, particularly around OpenSSL, crypto APIs, and package compatibility. The main challenges are legacy dependencies and deprecated Node.js APIs.

## Breaking Changes Analysis

### Node.js v16 → v18
- **OpenSSL 3.x upgrade** - Major crypto/TLS changes affecting legacy algorithms
- **Global `fetch()` API** - Experimental but available
- **Stricter crypto behaviors** - Some legacy crypto operations require `--legacy-openssl-provider`
- **V8 engine updates** - Performance improvements but potential compatibility issues

### Node.js v18 → v20
- **Built-in `fetch()` stable** - No longer experimental
- **Updated V8 engine** - Additional performance and security improvements
- **Stricter ES modules support** - Better but potentially breaking CommonJS interactions
- **Enhanced permission model** - `--experimental-permission` flag available

## Critical Dependencies Analysis

### 🚨 BLOCKING Dependencies (Must Upgrade)

#### 1. `mongoose@^4.10.1` → `mongoose@^8.x`
- **Issue**: Version 4.x incompatible with Node.js 18+
- **Breaking Changes**:
- Connection handling changes
- Deprecated callback patterns
- Schema validation changes
- **Migration Effort**: HIGH - Requires code refactoring

#### 2. `node-sass@^9.0.0` → `sass@^1.74.1` (Already Updated in React App)
- **Issue**: Native bindings incompatible with Node.js 18+
- **Status**: ✅ Backend still uses `node-sass`, frontend already migrated to `sass`
- **Action**: Migrate backend build process to use `sass`

#### 3. `fs-promise@^1.0.0` → Native `fs.promises`
- **Issue**: Package deprecated, incompatible with modern Node.js
- **Migration**: Replace with native `fs.promises` API

#### 4. `request-promise@^4.1.1` → `node-fetch` or `undici`
- **Issue**: Built on deprecated `request` package
- **Migration**: Replace with modern HTTP client

### ⚠️ HIGH PRIORITY Dependencies (Should Upgrade)

#### 1. `express@^4.14.0` → `express@^4.19.x`
- **Issue**: Security vulnerabilities in older versions
- **Compatibility**: Should work but needs testing

#### 2. `sequelize@^5.22.5` → `sequelize@^6.x`
- **Issue**: Deprecated APIs, better Node.js 20 support in v6
- **Migration Effort**: MEDIUM

#### 3. `connect-mongo@^2.0.3` → `connect-mongo@^5.x`
- **Issue**: Session store compatibility with newer Node.js versions

### 🔍 MODERATE PRIORITY Dependencies

#### Storybook Dependencies (React App)
- Currently on v7.x - consider upgrading to v8.x for better Node.js 20 support
- Most dependencies already compatible

## Code Pattern Analysis

### Deprecated Patterns Found

#### 1. Buffer Constructor Usage

// ❌ Deprecated pattern (if found)
Buffer(size)
Buffer(string, encoding)

// ✅ Modern pattern
Buffer.alloc(size)
Buffer.from(string, encoding)

#### 2. Crypto API Usage
- **Location**: `src/lib/ProofOfWorkOnLogin/md5.js`, `crypto-js` package usage
- **Risk**: Potential compatibility issues with OpenSSL 3.x
- **Action**: Test crypto operations thoroughly

#### 3. Callback-based Async Patterns
- **Location**: Throughout MongoDB operations with Mongoose v4
- **Action**: Migrate to Promise-based or async/await patterns

## Implementation Plan

### Phase 1: Preparation & Risk Mitigation (Week 1-2)

#### Step 1.1: Environment Setup

# Create Node.js version testing branch
git checkout -b node-upgrade-testing

# Install Node Version Manager if not present
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash

# Install Node.js versions for testing
nvm install 18.20.4  # Latest LTS v18
nvm install 20.15.1  # Latest LTS v20

#### Step 1.2: Dependency Audit

# Check for vulnerabilities
npm audit

# Check for outdated packages
npm outdated

# Analyze dependency tree
npm ls --depth=0

#### Step 1.3: Critical Dependency Upgrades

**Replace `fs-promise`:**

npm uninstall fs-promise
# Update code to use native fs.promises

**Replace `request-promise`:**

npm uninstall request-promise request
npm install node-fetch@^2.7.0  # or undici for Node.js 18+

**Upgrade Mongoose:**

npm install mongoose@^8.5.0
# Requires significant code changes - see migration guide

### Phase 2: Node.js 18 Migration (Week 3-4)

#### Step 2.1: Update Engine Requirements

// package.json
{
  "engines": {
    "node": "18",
    "npm": "9"
  }
}

#### Step 2.2: Test with Node.js 18

nvm use 18
npm install
npm run test:code:jest
npm run test:code:mocha
npm run build:app

#### Step 2.3: Handle OpenSSL Issues
If crypto issues arise:

# Temporary workaround for legacy crypto
export NODE_OPTIONS="--openssl-legacy-provider"

### Phase 3: Node.js 20 Migration (Week 5-6)

#### Step 3.1: Update to Node.js 20

// package.json
{
  "engines": {
    "node": "20",
    "npm": "10"
  }
}

#### Step 3.2: Leverage New Features
- Replace any remaining fetch polyfills with native `fetch()`
- Update test runners to use Node.js built-in test runner where applicable

#### Step 3.3: Performance Optimization
- Update npm scripts to leverage Node.js 20 performance improvements
- Review and optimize any ES module imports

## Testing Strategy

### 1. Unit Tests

# Run existing test suites
npm run test:code:jest
npm run test:code:mocha

### 2. Integration Tests

# Test all API endpoints
npm run test:e2e:accountflows
npm run test:e2e:leadflows
npm run test:e2e:otherflows

### 3. Build Verification

# Test all build processes
npm run build:app
npm run build:react-beehive
npm run build:onboarding-checklist

### 4. Performance Testing
- Monitor memory usage and startup times
- Compare performance metrics before and after upgrade
- Test under production-like load

## Risk Mitigation

### 1. Rollback Plan
- Maintain Node.js v16 Docker images
- Keep package-lock.json backups
- Document all configuration changes

### 2. Gradual Deployment
- Deploy to staging environment first
- Use feature flags for critical functionality
- Monitor error rates and performance metrics

### 3. Monitoring
- Set up enhanced logging for Node.js version-specific issues
- Monitor crypto operation performance
- Watch for deprecation warnings

## File-Specific Changes Required

### 1. `/checkNodeVersion.js`
Update to accept Node.js 18 and 20:

const expectedNodeVersion = process.env.NODE_ENV === 'production' ? '20' : ['18', '20'];

### 2. `/.nvmrc`
Update to specify Node.js 20:

20.15.1

### 3. Build Scripts
Update any hardcoded Node.js version references in:
- Docker files
- CI/CD configurations
- Deployment scripts

## Dependencies Migration Guide

### Mongoose v4 → v8 Migration
**Key Changes:**
- Connection string format changes
- Callback removal (use Promises/async-await)
- Schema validation updates
- Middleware signature changes

**Example Migration:**

// ❌ Old (v4)
mongoose.connect(uri, options, callback);

// ✅ New (v8)
await mongoose.connect(uri, options);

### Express Middleware Updates
**Helmet Configuration:**
- Update helmet configuration for compatibility
- Review CSP settings for new Node.js features

## Success Criteria

- [ ] All tests passing on Node.js 20
- [ ] No deprecation warnings in logs
- [ ] Performance metrics equal or better than Node.js 16
- [ ] All critical dependencies updated
- [ ] Production deployment successful
- [ ] No regression in functionality

## Post-Upgrade Considerations

### 1. Documentation Updates
- Update development setup documentation
- Revise deployment guides
- Update dependency management processes

### 2. Team Training
- Brief team on new Node.js features
- Document migration learnings
- Update coding standards for new Node.js version

### 3. Future Upgrades
- Establish regular Node.js upgrade schedule
- Monitor Node.js release cycle
- Proactive dependency management strategy

## Security Vulnerabilities to Address

Recent Express.js security releases (September 2024) identified several critical vulnerabilities that may affect your upgrade:

### High Severity
- **CVE-2024-45590** - body-parser DoS vulnerability (upgrade to ≥1.20.3)
- **CVE-2024-47178** - basic-auth-connect timing attack (upgrade to ≥1.1.0)

### Moderate Severity
- **CVE-2024-43796** - Express XSS via response.redirect() (upgrade to ≥4.20.0)
- **CVE-2024-43799** - send utility XSS vulnerability (upgrade to ≥0.19.0)
- **CVE-2024-43800** - serve-static XSS vulnerability (upgrade to ≥1.16.0)

**Action Required**: Verify these vulnerabilities are addressed during the upgrade process.

## Additional Gotchas and Challenges

### 1. Docker and Container Configuration

**Files to Update:**
- `Dockerfile` - Currently uses `FROM node:16-buster`
- `docker-compose.yml` - Service configurations
- `.gitpod.dockerfile` - Gitpod environment still references Node 10.16.2

**Required Changes:**

# Update Dockerfile
FROM node:20-bookworm  # or node:20-alpine for smaller size

# Update .gitpod.dockerfile
ENV NODE_ENGINE 20.15.1

### 2. Helmet.js Compatibility

Your app uses `helmet@^3.1.0` which is very outdated. Helmet v7+ has significant breaking changes:

**Current Issues:**
- Helmet v3 may not be compatible with Node.js 20
- Missing newer security headers
- Deprecated API usage

**Required Updates:**

npm install helmet@^7.1.0

**Code Changes Required:**

// OLD (v3)
app.use(helmet());

// NEW (v7) - explicit configuration may be needed
app.use(helmet({
  contentSecurityPolicy: {
    directives: {
      defaultSrc: ["'self'"],
      styleSrc: ["'self'", "'unsafe-inline'"],
      // ... other directives
    },
  },
}));

### 3. File Upload Security (multiparty)

Found usage of `multiparty@^4.1.3` in `/src/config/express/requests.js`:

**Potential Issues:**
- May have security vulnerabilities with newer Node.js versions
- Stream handling changes in Node.js 18+

**Recommended Actions:**
1. Audit multiparty usage for security issues
2. Consider migrating to `formidable` or `multer`
3. Test file upload functionality thoroughly after upgrade

### 4. Build Tool Compatibility Issues

**Gulp Compatibility:**
- Current Gulp version may have `primordials` errors with Node.js 18+
- Need to verify all Gulp plugins are compatible

**Node-gyp Issues:**
- Python version compatibility (requires Python ≥3.6, ≤3.11 for older node-gyp)
- Native addon compilation for bcrypt, node-sass
- Windows build tools compatibility

### 5. TypeScript Configuration Updates

**Current Setup Analysis:**
- Main project: ES2015 target, CommonJS modules
- React app: ES2017 target, ESNext modules
- Using older TypeScript compilation targets

**Recommended Updates:**

// Update tsconfig.json target for Node.js 20
{
  "compilerOptions": {
    "target": "ES2022",  // Node.js 20 supports ES2022
    "lib": ["ES2022"]
  }
}

### 6. Environment Variable Security

**High Usage of process.env Found:**
- 50+ files directly accessing `process.env`
- No environment variable validation

**Security Recommendations:**
1. Implement environment variable validation using `joi` or `zod`
2. Create centralized environment configuration
3. Avoid exposing sensitive variables in client-side code

### 7. CI/CD Pipeline Updates

**Files to Update:**
- `scripts/codeship/setup.sh` - May reference specific Node.js versions
- GitHub Actions workflows (if any)
- Heroku buildpacks configuration

### 8. Performance Considerations

**Potential Node.js 20 Benefits:**
- ~20% faster startup times
- Improved memory usage
- Better ES modules performance

**Monitoring Required:**
- Watch for memory usage patterns changes
- Monitor CPU usage for crypto operations
- Test WebAssembly performance if used

### 9. Third-Party Service Integrations

**Services Using Node.js-specific Code:**
- Onfido integration
- Revolut/Starling bank integrations
- Trustpilot API calls
- AWS SDK usage

**Action Required:**
- Test all third-party integrations after upgrade
- Verify API client compatibility
- Check for any Node.js version-specific workarounds

### 10. Testing Infrastructure Updates

**Browser Test Configuration:**
- Selenium container may need updates
- WebDriver compatibility with Node.js 20
- Update test runner configurations

## Extended Testing Strategy

### 1. Regression Testing Checklist
- [ ] File upload functionality (multiparty)
- [ ] All third-party API integrations
- [ ] Crypto operations (bcrypt, OTP generation)
- [ ] Database connections (MongoDB, PostgreSQL)
- [ ] Email sending functionality
- [ ] PDF generation (if applicable)
- [ ] Image processing operations

### 2. Performance Baseline Testing
- [ ] Measure startup time before/after
- [ ] Monitor memory usage patterns
- [ ] Test concurrent request handling
- [ ] Benchmark crypto operations
- [ ] Database query performance

### 3. Security Testing
- [ ] Verify Helmet.js headers are correct
- [ ] Test CSP policies still work
- [ ] Validate environment variable access
- [ ] Check for any new deprecation warnings

## Emergency Rollback Plan

### Quick Rollback Steps
1. **Docker Images**: Maintain Node.js 16 images in registry
2. **Package Locks**: Keep backup of `package-lock.json` files
3. **Environment Variables**: Document any changed NODE_OPTIONS
4. **Database**: Ensure no schema changes during upgrade
5. **Monitoring**: Set up alerts for error rate spikes

### Rollback Triggers
- Error rate increase >5%
- Memory usage increase >30%
- Any critical functionality failures
- Third-party integration failures

## BeeHive (Next.js) Integration Complexities

### **Hybrid Architecture Challenges**

Your application has a **unique hybrid setup** where Next.js BeeHive is served as static exports through Express.js middleware:

// Express serves static Next.js exports
app.use("/uk", csrfProtection, staticContentReactRoutes);

// Next.js config for static export
const nextConfig = {
  output: "export",  // Static site generation
  rewrites: async () => ({
    fallback: [{ source: "/:path*", destination: "http://localhost:3000/:path*" }]
  })
};

### **Critical Static Export Issues (Node.js 18+)**

1. **SubtleCrypto Edge Runtime Bug**: Node.js 18.17+ breaks `crypto.subtle.importKey()` in Next.js Edge Runtime
- **Impact**: Authentication/JWT processing may fail (need to verify any usage in codebase)
- **Mitigation**: Avoid crypto operations in middleware, use API routes instead

2. **Static Export Compatibility**: Next.js 14 removed `next export` command in favor of `output: 'export'`
- **Your Current Setup**: Already using `output: 'export'` ✅
- **Risk**: Serving mechanism through Express middleware needs validation

3. **Build System Integration**: Next.js static export must work with Express.js static file serving
- **File Path**: `react-beehive/out/` → Express middleware → Production serving
- **Testing Required**: Full build → export → Express integration pipeline

### **POC Upgrade Analysis Integration**

Based on PR #8786 findings, **additional complexities identified**:

#### **ESLint v9 Migration**

// New flat config requirement
import { FlatCompat } from '@eslint/eslintrc';
export default [/* flat config objects */];

#### **Build System ESM Conversion**

// gulpfile.js → gulpfile.mjs with complex ESM/CommonJS interop
import { createRequire } from "module";
const require = createRequire(import.meta.url);
const sass = require('gulp-sass')(require('node-sass')); // Still problematic

#### **Incomplete Dependencies in POC**
- `node-sass` still present (blocks ARM64 deployment)
- Mongoose migration not implemented
- `fs-promise` replacement missing

---

## Implementation Timeline

### **Phase 1: Infrastructure Preparation (Weeks 1-2)**
- [ ] **Week 1**: Validate BeeHive static export compatibility with Node.js 18
- [ ] **Week 1**: Test Heroku deployment pipeline with Node.js 18 base image
- [ ] **Week 2**: Complete node-sass → dart-sass migration (blocking for ARM64)
- [ ] **Week 2**: Replace deprecated packages (`fs-promise`, `request-promise`)

### **Phase 2: Core Dependency Updates (Weeks 3-5)**
- [ ] **Week 3**: Mongoose v4 → v8 migration (callback → Promise patterns)
- [ ] **Week 3**: ESLint v9 flat config migration
- [ ] **Week 4**: Express.js middleware and build system updates
- [ ] **Week 5**: Integration testing with BeeHive static export pipeline

### **Phase 3: Node.js 18 Upgrade (Weeks 6-7)**
- [ ] **Week 6**: Node.js v16 → v18 upgrade and compatibility testing
- [ ] **Week 6**: Docker image updates and CI/CD pipeline validation
- [ ] **Week 7**: Staging deployment and performance benchmarking

### **Phase 4: Node.js 20 Preparation (Weeks 8-10)**
- [ ] **Week 8**: Research Node.js 20 specific compatibility issues
- [ ] **Week 9**: Node.js v18 → v20 upgrade in development environment
- [ ] **Week 10**: Production deployment with full monitoring

---

## Risk Assessment

| Risk Category | Impact | Mitigation Strategy |
|---------------|---------|-------------------|
| **BeeHive Static Export** | **HIGH** | Dedicated integration testing, rollback to current serving method |
| **Build System (Gulp/ESM)** | **HIGH** | Incremental migration, maintain fallback build process |
| **SubtleCrypto Edge Runtime** | **MEDIUM** | Move crypto operations to API routes |
| **ESLint v9 Migration** | **MEDIUM** | Phased config migration, lint rule validation |
| **Database Layer (Mongoose)** | **HIGH** | Extensive testing, gradual callback → Promise conversion |

---

## Deployment Validation Checklist

### **BeeHive Integration Tests**
- [ ] Static export builds successfully with Node.js 18/20
- [ ] Express.js serves exported files correctly
- [ ] Client-side routing works in production
- [ ] Storybook builds and deploys properly

### **Performance Benchmarks**
- [ ] Build times remain within 10% of current
- [ ] Bundle sizes don't increase significantly
- [ ] Static file serving latency unchanged
- [ ] Memory usage stays within container limits

### **Security Validation**
- [ ] All high/critical CVEs resolved
- [ ] Crypto operations function correctly
- [ ] Authentication flows remain intact
- [ ] CSRF protection maintained

---

## Additional Resources

- [Node.js v18 Release Notes](https://nodejs.org/en/blog/release/v18.0.0)
- [Node.js v20 Release Notes](https://nodejs.org/en/blog/release/v20.0.0)
- [Next.js Static Export Guide](https://nextjs.org/docs/app/building-your-application/deploying/static-exports)
- [Next.js 14 Migration Guide](https://nextjs.org/docs/app/building-your-application/upgrading/version-14)
- [Mongoose v8 Migration Guide](https://mongoosejs.com/docs/migrating_to_8.html)
- [ESLint v9 Flat Config Migration](https://eslint.org/docs/latest/use/configure/migration-guide)
- [Express.js Security Best Practices](https://expressjs.com/en/advanced/best-practice-security.html)
- [Express.js September 2024 Security Releases](https://expressjs.com/2024/09/29/security-releases.html)
- [Helmet.js v7 Migration Guide](https://helmetjs.github.io/)
- [Node-gyp Troubleshooting Guide](https://github.com/nodejs/node-gyp/blob/main/docs/README.md)