# dataframe interchange protocol A way to convert one type of dataframe into another type (ex. cuDF to Vaex) <!-- Put the link to this slide here so people can follow --> <!-- slide: https://hackmd.io/p/template-Talk-slide --> --- Goal: writing code that can accept any type of dataframe :tada: --- ## Scope of the internship work - implement the protocol for Vaex - contribute to `veax/pandas` :heart: - develop an example notebook - write a blog post :writing_hand: --- ### Vaex is a high performance Python library for lazy Out-of-Core DataFrames (similar to Pandas), to visualize and explore big tabular datasets. authors: Maarten Breddels, Jovan Veljanoski --- Idea for vaex - astronomy - graphic representation --- ## Implementation for `vaex` library - defines transformation from general `__dataframe__` class to `vaex` dataframe class - defines `__dataframe__` attributes for `vaex` dataframe --- ## The process - iterate through the colomns in the dataframe - define buffer size, location and values - transfer data back to numpy column - join all numpy arrays to vaex dataframe --- ```mermaid graph LR df["__dataframe__"<br/> <br/>_DataFrame<br/>_Column<br/>_Buffer] ``` --- ```mermaid graph LR df_vaex[Vaex dataframe]-.-> |interchange protocol|__dataframe__ ``` --- ```mermaid graph LR df_vaex[Vaex dataframe]-.-> |interchange protocol|__dataframe__ __dataframe__ --> |"from_dataframe()"|df_vaex[Vaex dataframe] ``` --- ## from_dataframe() ```mermaid graph LR subgraph one[__dataframe__<br/>] direction TB A[dataframe] --> B[df_column] B --> C["buffer"] end subgraph two[Vaex] direction BT D[column] D --> E[Vaex dataframe] end C -.-> D ```