HackMD
  • New!
    New!  “Bookmark” and save your note
    Find a note that's worth keeping or want reading it later? “Bookmark” it to your personal reading list.
    Got it
    • New!  “Bookmark” and save your note
      New!  “Bookmark” and save your note
      Find a note that's worth keeping or want reading it later? “Bookmark” it to your personal reading list.
      Got it
      • Options
      • Versions and GitHub Sync
      • Transfer ownership
      • Delete this note
      • Template
      • Insert from template
      • Export
      • Dropbox
      • Google Drive
      • Gist
      • Import
      • Dropbox
      • Google Drive
      • Gist
      • Clipboard
      • Download
      • Markdown
      • HTML
      • Raw HTML
      • ODF (Beta)
      • Sharing Link copied
      • /edit
      • View mode
        • Edit mode
        • View mode
        • Book mode
        • Slide mode
        Edit mode View mode Book mode Slide mode
      • Note Permission
      • Read
        • Owners
        • Signed-in users
        • Everyone
        Owners Signed-in users Everyone
      • Write
        • Owners
        • Signed-in users
        • Everyone
        Owners Signed-in users Everyone
      • More (Comment, Invitee)
      • Publishing
        Everyone on the web can find and read all notes of this public team.
        After the note is published, everyone on the web can find and read this note.
        See all published notes on profile page.
      • Commenting Enable
        Disabled Forbidden Owners Signed-in users Everyone
      • Permission
        • Forbidden
        • Owners
        • Signed-in users
        • Everyone
      • Invitee
      • No invitee
    Menu Sharing Help
    Menu
    Options
    Versions and GitHub Sync Transfer ownership Delete this note
    Export
    Dropbox Google Drive Gist
    Import
    Dropbox Google Drive Gist Clipboard
    Download
    Markdown HTML Raw HTML ODF (Beta)
    Back
    Sharing
    Sharing Link copied
    /edit
    View mode
    • Edit mode
    • View mode
    • Book mode
    • Slide mode
    Edit mode View mode Book mode Slide mode
    Note Permission
    Read
    Owners
    • Owners
    • Signed-in users
    • Everyone
    Owners Signed-in users Everyone
    Write
    Owners
    • Owners
    • Signed-in users
    • Everyone
    Owners Signed-in users Everyone
    More (Comment, Invitee)
    Publishing
    Everyone on the web can find and read all notes of this public team.
    After the note is published, everyone on the web can find and read this note.
    See all published notes on profile page.
    More (Comment, Invitee)
    Commenting Enable
    Disabled Forbidden Owners Signed-in users Everyone
    Permission
    Owners
    • Forbidden
    • Owners
    • Signed-in users
    • Everyone
    Invitee
    No invitee
       owned this note    owned this note      
    Published Linked with
    Like BookmarkBookmarked
    Subscribed
    • Any changes
      Be notified of any changes
    • Mention me
      Be notified of mention me
    • Unsubscribe
    Subscribe
    --- title: Lab Meeting Minutes 2020/08/18 tags: lab_meeting --- >Outline >[TOC] --- # PERAL Lab Meeting - 時間:109年8月18日 19:00 - 地點:暨南大學科三館321教室 - 主講人:田蕙瑜 - 出席者:吳坤熹老師 (remote)、黃晨瑋、謝萬霖、劉又聖、陳嘉瑋、劉怡君 - 會議主題:Brief Introduction of RDMA (Remote Direct Memory Access) ## 會議內容 - RDMA(Remote Direct Memory Access) 1. RDMA(Remote Direct Memory Access)原理介紹,如何實現此技術。 - 傳統資料傳輸會先進行硬碟 I/O,再複製到 Socket buffer,過程中經歷了四次的 kernel / user 身分的切換,嚴重影響到效能 3. 摘要二 ## 問題討論 1. P.4 的 Infiniband網路,非眾所周知。應該要介紹一下。 2. P.6 透過 socket buffer 有什麼不好? - Normally the buffer size is small. When it is full, the application stops sending data. When it is empty, the receiver becomes idle, waiting for data to come. This mechanism seldom delivers data in "full speed", let alone copying between kernel/user space is an overhead which introduces extra delay. - In modern FreeBSD versions the default sizes of the socket receiving and sending buffers are respectively **64K** and **32K**. You may [change them on the fly](http://nginx.org/en/docs/freebsd_tuning.html) using the sysctls `net.inet.tcp.recvspace` and `net.inet.tcp.sendspace`. 4. P.11 的 HCA 是什麼的縮寫?所有縮寫在第一次出現時都應該列出全名。你在下一頁 (P.12) 解釋了全名是 Host Channel Adapter, 這是否表示兩頁的次序應該對調? 5. P.15 InfiniBand v2 如果透過 UDP 來傳資料,那就落入傳統的 UDP socket 傳送的缺點,會佔用 CPU time。這樣如何能跳過 kernel? * 列入待追蹤事項 6. P.16 理論上RDMA確實可以加快資料的傳輸。但能快多少?通常這時要透過一個實驗,展示其具體的效能改善數據。 - 40Gbps的TCP/IP流能耗盡主流伺服器的所有CPU資源;RDMA則解決了傳統TCP/IP通訊的技術痛點。例如,在40Gbps場景下,CPU佔用率從100%下降到5%,網路延時從ms級降低到10us以下。 #### 建議 1. 參考影片 [YouTube] [What is RDMA?](https://www.youtube.com/watch?v=lu78_C-9jvA) [name= Solomon] 2. 參考影片 [YouTube] [RDMA and Why It Matters](https://www.youtube.com/watch?v=fh2WkfXvzHM) [name=Solomon] 3. 參考影片 [YouTube] [HowTo Configure NFS over RDMA (RoCE)](https://www.youtube.com/watch?v=vKZqmZA4kIA) [name=Solomon] 4. 參考文獻 [RFC 5040](ftp://ftp.ncnu.edu.tw/rfc/rfc5040.txt) - A Remote Direct Memory Access Protocol Specification [name=Solomon] 5. 主講者應提早到現場設定環境及測試設備。像今天等大家到齊了還拖到 19:20 才開始,浪費大家時間,很不應該。 [name=Solomon] 6. 以後嚴格要求一週前須上傳簡報檔初稿。之後還有一週可以慢慢改進,但初稿沒上傳,今日不准走。嘉瑋說檔案在住的地方,很顯然就是沒有做到實驗室要求的 Backup。以後每天晚上工作結束時,都該養成把檔案先備份的習慣。 [name=Solomon] --- #### 問題 1. HCA 是建立在HOST Memory上 [name= 又聖] - Ans: [name= 蕙瑜] 2. 問 [name= 萬霖] - Ans: [name= 蕙瑜] 3. Application的Buffer、 [name=以恒] 4. Bypassing the Operating system可以解釋一下原理嗎? [name=嘉瑋] - Ans:可以跳過kernel不需經它處理 [name= 蕙瑜] 5. Registered memory 在註冊時不論要進行send還是receive的動作仍會同時建立SQ跟RQ? [name=怡君] - Ans: Yes. [name= 蕙瑜] 7. Completion Queue 的作用,能不能解釋一下 [name=Abner] - Ans:可以控制Send Quene、Receive Quene [name= 蕙瑜] ## 臨時動議 1. 又聖有關專利申請的報告,盡量提前。故接下來報告次序調整為嘉瑋、又聖、怡君。 [name= 萬霖] --- 散會: 結束時間:21:00

    Import from clipboard

    Advanced permission required

    Your current role can only read. Ask the system administrator to acquire write and comment permission.

    This team is disabled

    Sorry, this team is disabled. You can't edit this note.

    This note is locked

    Sorry, only owner can edit this note.

    Reach the limit

    Sorry, you've reached the max length this note can be.
    Please reduce the content or divide it to more notes, thank you!

    Import from Gist

    Import from Snippet

    or

    Export to Snippet

    Are you sure?

    Do you really want to delete this note?
    All users will lost their connection.

    Create a note from template

    Create a note from template

    Oops...
    This template is not available.
    All
    • All
    • Team
    No template found.

    Create a template

    Delete template

    Do you really want to delete this template?

    This page need refresh

    You have an incompatible client version.
    Refresh to update.
    New version available!
    See releases notes here
    Refresh to enjoy new features.
    Your user state has changed.
    Refresh to load new user state.

    Sign in

    Forgot password

    or

    By clicking below, you agree to our terms of service.

    Sign in via Facebook Sign in via Twitter Sign in via GitHub Sign in via Dropbox Sign in via Google

    New to HackMD? Sign up

    Help

    Documents

    Tutorials
    YAML Metadata
    Slide Example
    Book Example

    Contacts

    Talk to us
    Report an issue
    Send us email

    Cheatsheet

    Example Syntax
    Header # Header
    • Unordered List
    - Unordered List
    1. Ordered List
    1. Ordered List
    • Todo List
    - [ ] Todo List
    Blockquote
    > Blockquote
    Bold font **Bold font**
    Italics font *Italics font*
    Strikethrough ~~Strikethrough~~
    19th 19^th^
    H2O H~2~O
    Inserted text ++Inserted text++
    Marked text ==Marked text==
    Link [link text](https:// "title")
    Image ![image alt](https:// "title")
    Code `Code`
    var i = 0;
    ```javascript
    var i = 0;
    ```
    :smile: :smile:
    Externals {%youtube youtube_id %}
    LaTeX $L^aT_eX$

    This is a alert area.

    :::info
    This is a alert area.
    :::

    Versions

    Versions and GitHub Sync

    Sign in to link this note to GitHub Learn more
    This note is not linked with GitHub Learn more
     
    Add badge Pull Push GitHub Link Settings

    Version named by    

    More Less
    • Edit
    • Delete

    Note content is identical to the latest version.
    Compare with
      Choose a version
      No search result
      Version not found

    Feedback

    Submission failed, please try again

    Thanks for your support.

    On a scale of 0-10, how likely is it that you would recommend HackMD to your friends, family or business associates?

    Please give us some advice and help us improve HackMD.

     

    Thanks for your feedback

    Remove version name

    Do you want to remove this version name and description?

    Transfer ownership

    Transfer to
      Warning: is a public team. If you transfer note to this team, everyone on the web can find and read this note.

        Link with GitHub

        Please authorize HackMD on GitHub

        Please sign in to GitHub and install the HackMD app on your GitHub repo. Learn more

         Sign in to GitHub

        HackMD links with GitHub through a GitHub App. You can choose which repo to install our App.

        Push the note to GitHub Push to GitHub Pull a file from GitHub

          Authorize again
         

        Choose which file to push to

        Select repo
        Refresh Authorize more repos
        Select branch
        Select file
        Select branch
        Choose version(s) to push
        • Save a new version and push
        • Choose from existing versions

        Pull from GitHub

         
        File from GitHub
        File from HackMD

        GitHub Link Settings

        File linked

        Linked by
        File path
        Last synced branch

        Danger Zone

        Unlink
        You will no longer receive notification when GitHub file changes after unlink.

        Syncing

        Push failed

        Push successfully