HackMD
Technology Stack Selection for high speed 3D rendering
Introduction
I'm aiming to develop a mobile application in which requires the realtime 3D rendering from the UDP packet that flows in 60fps. In this phase, I would like to discuss about how should I define the appropriate tech stacks from several perspective.
Identifying Prerequisites
Business persepective
The first thing we have to work on is about business plans, since the capability of the application will affect out development phase, in cost, delivery, and quality. We should decide with careful manners.
Currently, we are in the Proof of Concept phase, so we are more likely to be able to limit the range of devices to support, which means we can solely develop for iOS devices.
We still have advantages if we choose languages or frameworks in which mak us easy to develop for both iOS and Android, therefore, let's analyze the pros and cons about them.
Framework Comparison for Dashboard App
Since our project involves real-time data processing and 3D visualization, we could consider using game engines like Unreal Engine or Unity. However, for a dashboard with a single 3D visualization window, these solutions would be overkill. Therefore, we should focus on native solutions instead.
Main Requirements
- Real-time UDP packet processing (60fps)
- 3D visualization of car
- Dashboard UI with metrics
- iOS platform
Choosing the framework
Here's a detailed comparison table with characteristics and trade-offs for Swift, Flutter, and React Native.
| Framework | Characteristics | Strengths | Weaknesses |
|---|---|---|---|
| Swift | - Direct access to native APIs (Metal, SceneKit, Core Animation). - Optimized for real-time processing & 3D rendering. - No extra overhead from bridging layers. |
✅ Best performance for iOS-native apps. ✅ Lowest latency (ideal for real-time UDP processing and GPU-heavy apps). ✅ Strong stability with full system integration. |
❌ iOS-only, limiting cross-platform expansion. ❌ Requires dedicated iOS engineers ❌ UI development speed can be slower than Flutter’s hot reload. |
| Flutter | - Uses Dart and the Skia rendering engine, making it able to develop for cross-platform. - Provides a single codebase for iOS & Android. - Features hot reload for fast UI development. - Uses a Platform Channel to interact with native code. |
✅ Faster UI development with hot reload. ✅ Cross-platform (iOS, Android, Web, Desktop). ✅ Rich UI customization independent of native components. ✅ Lower development cost compared to native iOS-only solutions. |
❌ Platform channel adds latency, making it less suitable for real-time processing. ❌ UI performance slightly lower than native due to additional rendering layers. ❌ Limited enterprise adoption in iOS-heavy environments. |
| React Native | - Uses JavaScript & React, making it familiar to web developers to create cross platform applications. - Renders native components via JS Bridge. - Supports third-party libraries for native modules. |
✅ Reuses JavaScript skills, reducing learning curve for web developers. ✅ Cross-platform, with a large ecosystem of libraries. ✅ Decent UI performance for non-intensive apps. |
❌ JavaScript bridge introduces latency, making it unsuitable for real-time applications. ❌ Performance issues with complex UI & animations. Not ideal for CPU/GPU-intensive tasks like 3D rendering. |
| Framework | Architecture Layers |
|---|---|
| Swift | UDP → C++/Swift Processing → Metal/SceneKit → UI |
| Flutter | UDP → Native → Platform Channel → Dart → Skia → UI |
| React Native | UDP → Native → Bridge → JS → UI |
Quality and Performance
For our real-time visualization app requiring 60 FPS UDP processing and 3D rendering, Swift + SceneKit is the superior choice. While Flutter and React Native offer cross-platform development, their bridging overhead (1-2ms for Flutter, 2-3ms for React Native) makes stable 60 FPS difficult to achieve.
Compared to Metal, SceneKit is more suitable as it provides a high-level abstraction for simple 3D rendering, reducing development complexity while still leveraging hardware-accelerated graphics. Metal, though powerful, demands low-level shader programming and manual resource management, which would be overkill for our needs.
With Swift + SceneKit, we get direct socket control, low-latency GPU access (0.1-0.5ms per frame), and lower memory consumption (50-100MB vs. 100-200MB for alternatives). This ensures a smoother real-time experience, preventing frame drops and latency spikes that could impact usability.
Software Architecture Overview
After evaluating our requirements, we have determined that developing exclusively for iOS is sufficient from a business perspective. Given this, while Flutter offers faster UI development, its additional processing layers hinder real-time performance. React Native is the least suitable due to its reliance on JavaScript, which introduces significant delays. Therefore, Swift remains the optimal solution for ensuring stability, performance, and seamless integration with iOS.
Additionally, For real-time UDP packet processing, the performance difference between C++ and Swift is minimal (C++: ~0.01-0.02ms per packet, Swift: ~0.02-0.04ms per packet). Since Swift eliminates the need for a bridging layer, it simplifies integration while maintaining low-latency processing. Therefore, we have chosen Swift over C++ for handling UDP packets.
Selected Architecture
| Framework | Architecture Layers |
|---|---|
| Swift | UDP → Swift Processing → SceneKit → UI |
For further details, refer to the following table:
Detailed Performance Analysis
| Framework | UDP Processing | Data Transfer | 3D Rendering | System Resources |
|---|---|---|---|---|
| Swift (C++) | - Native socket performance - ~60fps packet processing - Direct memory management - Hardware-level optimization |
- Small overhead from C++ bridge - ~0.1-0.5ms bridge delay - Binary data handling - Zero-copy possible |
- Direct Metal access - ~60fps rendering - Hardware acceleration - Native shader compilation |
- Manual memory control with ~50-100MB baseline - Low overhead with ~10-15% CPU for UDP - Efficient thread usage with ~20-30% CPU for rendering - Predictable resource allocation |
| Swift (Native) | - Native Network.framework - ~60fps packet processing - ARC memory management - Platform optimizations |
- No bridging needed - Direct data access - Swift value types - Zero overhead |
- Direct Metal access - ~60fps rendering - Swift Metal API - Native compilation |
- ARC handling with ~70-120MB baseline - Moderate CPU usage at ~15-20% for UDP - ~20-30% CPU for rendering with auto thread management - Slightly higher memory overhead than C++ |
| Flutter | - Native implementation required - Platform channel overhead - ~30-50fps effective - Dart isolate limitations |
- Channel serialization - ~1-2ms channel delay - JSON/binary conversion - Copy overhead |
- Skia engine overhead - ~30-40fps for 3D - OpenGL limitations - Widget tree updates |
- Higher baseline memory at ~100-150MB with Dart VM - Increased CPU load at ~20-25% for UDP - ~30-40% CPU for rendering with isolate overhead - Notable GC impact on performance |
| React Native | - Native module needed - Bridge performance impact - ~30-40fps effective - JS engine limitations |
- Bridge serialization - ~2-3ms bridge delay - JSON conversion - Multiple copies |
- Native module for 3D - ~30fps for 3D - Bridge overhead - JS render loop |
- Highest memory usage at ~120-200MB - Heavy CPU utilization at ~25-30% for UDP - ~35-45% CPU for rendering - Additional overhead from JS engine and bridge |
| Framework | UDP Processing | Data Transfer | 3D Rendering | System Resources |
|---|---|---|---|---|
| Swift (C++) | - Native socket performance - ~60fps packet processing - Direct memory management - Hardware-level optimization |
- Small overhead from C++ bridge - ~0.1-0.5ms bridge delay - Binary data handling - Zero-copy possible |
- Direct Metal access - ~60fps rendering - Hardware acceleration - Native shader compilation |
- Manual memory control with ~50-100MB baseline - Low overhead with ~10-15% CPU for UDP - Efficient thread usage with ~20-30% CPU for rendering - Predictable resource allocation |
| Swift (Native) | - Native Network.framework - ~60fps packet processing - ARC memory management - Platform optimizations |
- No bridging needed - Direct data access - Swift value types - Zero overhead |
- Direct Metal access - ~60fps rendering - Swift Metal API - Native compilation |
- ARC handling with ~70-120MB baseline - Moderate CPU usage at ~15-20% for UDP - ~20-30% CPU for rendering with auto thread management - Slightly higher memory overhead than C++ |
| Flutter (OpenGL) | - Native implementation required - Platform channel overhead - ~30-50fps effective - Dart isolate limitations |
- Channel serialization - ~1-2ms channel delay - JSON/binary conversion - Copy overhead |
- Skia engine overhead - ~30-40fps for 3D - OpenGL limitations - Widget tree updates |
- Higher baseline memory at ~100-150MB with Dart VM - Increased CPU load at ~20-25% for UDP - ~30-40% CPU for rendering with isolate overhead - Notable GC impact on performance |
| Flutter (Vulkan) | - Native implementation required - Vulkan FFI integration - ~50-60fps packet processing - Low-level control |
- Direct C++ bridge to Vulkan engine - ~0.5-1ms overhead - Efficient binary handling - Zero-copy memory model |
- Direct Vulkan API - ~60fps rendering - Hardware acceleration - Optimized GPU usage |
- Vulkan optimized memory management - ~80-130MB baseline memory - ~15-20% CPU for UDP processing - ~25-35% CPU for rendering - Efficient multi-threaded GPU access |
| React Native | - Native module needed - Bridge performance impact - ~30-40fps effective - JS engine limitations |
- Bridge serialization - ~2-3ms bridge delay - JSON conversion - Multiple copies |
- Native module for 3D - ~30fps for 3D - Bridge overhead - JS render loop |
- Highest memory usage at ~120-200MB - Heavy CPU utilization at ~25-30% for UDP - ~35-45% CPU for rendering - Additional overhead from JS engine and bridge |
