# VR Streaming Research

# VR Player
Original Repo: https://github.com/wheat7/VRPlayer/

## How to Setup
1. Download and install android studio: https://developer.android.com/studio/install
2. Open the file (File > Open, select the folder)
3. Compile: 
4. Use emulator or plug in your phone with developer mode
5. Dependencies are declared in build.gradle

** Use ijk player as dependency

## Architecture in VR player
- Key function to determine src urI in ijkplayer: `setDataSource()`


- Idea: replace *`path`* with web urI to play mp4 video from the web

### *Reference*
* https://juejin.im/post/5bd11ca7e51d457a32545f5e
* https://www.jianshu.com/p/daf0a61cc1e0
* https://zhuanlan.zhihu.com/p/45251444
* https://www.itread01.com/content/1537765196.html

## Experiment: Replace `path` with custom urI
**Result: Success**
- video url: http://vod.cntv.lxdns.com/flash/mp4video61/TMS/2017/08/17/63bf8bcc706a46b58ee5c821edaee661_h264818000nero_aac32-5.mp4
- Before: In `mPlayerView.java`

    public void setVideoPath(String path) {
        setVideoURI(Uri.parse(path));
    }

- After: In `mPlayerView.java`

    public void setVideoPath(String path) {
        setVideoURI(Uri.parse("http://vod.cntv.lxdns.com/flash/mp4video61/TMS/2017/08/17/63bf8bcc706a46b58ee5c821edaee661_h264818000nero_aac32-5.mp4"));
    }

- demo: https://drive.google.com/file/d/1Cf7v5L8RENYuxPDCdJ8LfzWWM9nBFZaq/view?usp=sharing
- Conclusion: ijkplayer support playing mp4 video from the web

### *Reference*
- [Android 超好用的播放器——ijkplayer - 简书](https://www.jianshu.com/p/c5d972ab0309)
- [Android ijkplayer的使用解析及播放器的制作 - github_2011的博客 - CSDN博客](https://blog.csdn.net/github_2011/article/details/77385498)

## Adding an interface to enter urI
### Result:

**Github link:**

### Implementation
- Added an `InputUriActivity` on startup instead of directly going to `VRListActivity`
- When `Submit` is pressed, pass the urI entered to `VRPlayerActivity` for steaming
- Preserve option to list local videos (`VRListActivity`) to compare the performance between local videos and online videos

public class InputUriActivity extends AppCompatActivity {

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_input_uri);
    }

    public void submitUri(View view) {
        EditText editText = (EditText) findViewById(R.id.editText);
        String message = editText.getText().toString();
        Intent intent = new Intent(this, VRPlayerActivity.class);
        intent.putExtra("path", message);
        startActivity(intent);
    }

    public void listLocalVideo(View view) {
        Intent intent = new Intent(this, VRListActivity.class);
        startActivity(intent);
    }
}

- *Reference*: https://developer.android.com/training/basics/firstapp/building-ui

## Bug 1: Fail to resolve hostname [Solved]
- **Behavior of the app**

- **Log message**

    2019-08-16 23:19:01.132 22721-23460/com.wheat7.vrplayer I/tv.danmaku.ijk.media.player.IjkMediaPlayer: onNativeInvoke 1
    2019-08-16 23:19:01.132 22721-23460/com.wheat7.vrplayer E/IJKMEDIA: Failed to resolve hostname vod.cntv.lxdns.com: No address associated with hostname
    2019-08-16 23:19:01.133 22721-23460/com.wheat7.vrplayer I/tv.danmaku.ijk.media.player.IjkMediaPlayer: onNativeInvoke 2
    2019-08-16 23:19:01.133 22721-23460/com.wheat7.vrplayer E/IJKMEDIA: http://vod.cntv.lxdns.com/flash/mp4video61/TMS/2017/08/17/63bf8bcc706a46b58ee5c821edaee661_h264818000nero_aac32-5.mp4: I/O error

- **Cause:** No internet permission
- **Solution:** request user permission in `AndroidManifest.xml`

        ...
  <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
  <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
  <uses-permission android:name="android.permission.INTERNET" />
        ...

## Bug 2: Protocol not found
- **Behavior of the app**

- **Log message**

    2019-09-10 08:12:29.652 24362-25689/com.wheat7.vrplayer W/IJKMEDIA: https protocol not found, recompile FFmpeg with openssl, gnutls or securetransport enabled.
    2019-09-10 08:12:29.652 24362-25689/com.wheat7.vrplayer E/IJKMEDIA: https://cdn.udn.com/media/video/10093120_720p_2560.mp4: Protocol not found
    2019-09-10 08:12:29.652 24362-25689/com.wheat7.vrplayer I/IJKMEDIA: SDL_JNI_DetachThreadEnv: [25689]
    2019-09-10 08:12:29.652 24362-25687/com.wheat7.vrplayer D/IJKMEDIA: FFP_MSG_ERROR: 0

- **Cause**
- `Protocol not found`
- Ijkplayer does not support `https` protocol
- **Solution**
- Recompile FFmpeg with openSSL.
- Tutorial: http://tcking.github.io/2016/02/25/compile-ijkplayer-with-openssl/

# 360 Player
Original repo: https://github.com/arnerak/360transitions

## Analysis of Code
**Report**: [360 player code analysis](https://docs.google.com/document/d/17yBbPMqD0F2G6lQgz00FVsrqbWDd2U3Zy-SBQP9XYGY/edit)

Trace the source code and determine which part of code handle the function below:
1. parse the file path and config
1. fetch data from HTTP server
2. determine the tile quality based on the measured bandwidth
3. read and decode tiles
4. track head orientation

## Tile Quality Control and Download
**Goal:** find method to predict head movement and download the visible tiles only.

**Report**: [360 player code analysis - Tile Quality Control and Download](https://docs.google.com/document/d/17yBbPMqD0F2G6lQgz00FVsrqbWDd2U3Zy-SBQP9XYGY/edit#heading=h.ewgsckuuu3vy)

**Suggestion**
1. Adapt our prediction algoritms into `predictTileVisibility()`
2. Add only highly visible tiles to `tileDownloadOrder`, decrease `numTiles`

# TOUCAN VR
Original repo:
- https://github.com/UCA4SVR/TOUCAN-VR
- https://github.com/UCA4SVR/TOUCAN-preprocessing
- https://github.com/UCA4SVR/TOUCAN_VR_parametrizer

## Architecture
這個project 由三個部分組成 :
- TOUCAN-VR Preprocessing
- https://github.com/UCA4SVR/TOUCAN-preprocessing
- 裡面的內容是一些由java寫成的script，用來下指令給FFMPEG和MP4Box
- 這個 project 的功能是將regular 360 video 切成有 tile 的360 video

- TOUCAN-VR parametrizer
- https://github.com/UCA4SVR/TOUCAN_VR_parametrizer
- 這個project是用java寫成的APP，可以用android studio裝在android手機上
- 主要功能是設定&傳送parameter給TOUCAN-VR，使TOUCAN-VR正常運作
- TOUCAN-VR
- https://github.com/UCA4SVR/TOUCAN-VR
- 這個project是用java寫成的APP，可以用android studio裝在android手機上
- 主要功能是接收parametrizer的parameter，並播放VR影像

## How to Use
使用流程 :
1. 先用 TOUCAN-VR Preprocessing 搭配xml檔進行360 video 的轉換，並將轉換好的結果放到http server上，以便提供URL給TOUCAN-VR parametrizer
2. 設定xml檔，TOUCAN-VR parametrizer 會從中獲得URL及其他設定，並呼叫TOUCAN-VR進行撥放
3. TOUCAN-VR進行影片撥放

PS : xml檔github裡面都有範例

**Original Document:** [TOUCAN_VR Document](/9_q-TXiyRfqPKKaqSWRWFA)

## Analysis of Code
**Report**: [TOUCAN_VR code analysis](https://docs.google.com/document/d/1Lm-VlT7vjTlCqHJ_lcG7riGYsuzeykfQOguyOluAnn8/edit?usp=sharing)

Trace the source code and determine which part of code handle the function below:
1. The player
1. fetch data from HTTP server
2. determine the tile quality based on the measured bandwidth
3. read and decode tiles
4. track head orientation

## Tile Quality Control and Download
**Goal:** find method to predict head movement and download the visible tiles only.

- Tile quality control: `Minimal360Video.java` in TOUCAN_VR_parametrizer
- tiling
- prepare the scene
- Fail to determine function which download tiles from the web :(