# I Beta Tested OpenAI Codex on a ML problem

... and the result is impressive but it is nowhere near going to replace actual programmers.


First of all, I would like to thank the OpenAI team to have granted me access to the closed Beta.

In order to have a taste of how OpenAI Codex perfom at the task of translating english to python code, I did a machine learning project consisting of **predicting Bitcoin price** from Yahoo! Finance.

The work plan is rather simple, and i am going to use google colab (for running code) and openAI platform (for using Codex) side-by-side to achieve it:

- Extract Bitcoin time series data from an API
- Translate this data into training, validation, and test datasets
- Use Tensorflow and Keras to train a reccurent neural network on it.
- Visualize results.

These are the basic steps of every data science projects, and it will be interesting to see which steps OpenAI Codex perform best on, and make comments about its mistakes along the way.

Here is what you come across when beta testing OpenAI Codex:


## I - Extract Bitcoin time series data from an API
I copy paste both the command text and the python code generated by the command to a cell within a Google Colab notebook. Here is the first step done in a glimpse of an eye by Codex:


It is noteworthy to see that Codex not only extracted the data from the given link in the command but that it imported some libraries, including pandas that is necessary.
Other imported libraries like numpy, matplotlib, and seaborn are not useful.

We have imported the raw data, let's see what it looks like.


Surprisingly, Codex generated several functions doing the same thing but with a few variations.

The first one implemented is the most basic we could have expected.


The third one for example adds a few details :+1: .


**Remarks**

I manually changed the size of the figure within the functions.

Each proposed function adds exactly one new feature to the previous one.

The name of the proposed functions get longer and longer as more details are added to the plotting.

## II - Translate raw data into train/val/test datasets

We are going to use only Close price columns to make predictions.

Thus, we need to refactor it

... and to normalize it.

Here Codex is doing more that what i asked and this is problematic, i need to change the code manually.


A crucial part is batching the time serie, and separate into train and test sets.The generated code by Codex is pretty bad, the names of arguments are unclear, and the asserts are terrible.


Here, it is faster to write my own code :-1: .

## III - Deep Learning training

The creation of the model is perfect, I only changed the activation who was softmax, and the loss who originally was generated as a categorical cross entrepy loss.



This model's architecture is similar to the one I had in my mind when chosing to cope with this prediction problem :+1: .

Let's train it.

![Uploading file..._rnnuqayp1]()

To obtain this, I did not write another command, I only executed Codex once more to generate some more code. It was logical that Codex would write the training part even if not asked explicitly :+1: .

## IV - Visualize results

... This was also generated after.


The results are plotted with the previously generated code:



The results are good enough for this example.

**Remark**
If I try once more to execute Codex, it only proposes me to save the model and nothing different after.

## V - Conclusion

To conclude, we saw that I gained a lot of time during part I, III, and IV of the project. However, it was a lot easier to do part II by myself.
Code here : https://github.com/AntoninLagarrigue/OpenAI-Codex-Predicts-Bitcoin

I think OpenAI Codex is a great tool for developers including data scientists. I used it as a tool, nothing more than that. I described the tasks Codex had to achieve in a rather low-level format, hence you won't do much if you don't have a really clear idea of what you want to code and the precise steps to achieve it. So there is no way it will replace humans at coding :).

However, what it almost certainly will do, though, is make their jobs far easier. By speeding the coding process, Codex can free up time to focus on higher-value tasks, like figuring out the technical solution to a new business problem.