Weekly Findings In Data Science and AI
A New AutoML Model With Mind-Boggling Results — Weekly Findings
This is a summary of interesting findings in data science and AI I’ve discovered recently. Hopefully, this can be a recurring type of article that I can publish every week. Let’s get into it.
The Incredible AutoML model
A tabular classification model called TabPFN was recently published and the results are absolutely mind-boggling. In summary:
- Can be run (i.e. be trained and predict) in less than one second (!) with a GPU
- Requires no hyperparameters
- Beats hyperparameter-tuned gradient boosting given time constraints on a benchmark of datasets (up to 2000 samples in each dataset)
- Is of relatively small size with less than 26M parameters
- Bayesian and thus models uncertainty
The only catch is that it can only take up to 1k points, 100 features and 10 classes.
It is a meta-learned transformer model trained on synthetic data to perform approximate Bayesian inference. It takes as input training data (features and labels) and test data (without labels) and then outputs predictions/probabilities in a single forward pass (i.e. no tuning with backpropagation). According to the authors, it is unlikely to overfit.
This could be a game-changer. The limited number of points is of course a large constraint, but this is a great step in the right direction. It is likely that future improvements will develop very quickly. Not only could this simplify model selection for data scientists, but also its speed and small size means it can be applied in many circumstances online without supervision, such as on an embedded device. We’ve seen pre-trained models for images and text, but perhaps these will now become more prominent in the area of tabular data. Personally, I’m very excited about this.
Sources:
- Tweet presenting it by one of the authors
- Google colab notebook where you can try it out yourself
- GitHub page
- Blog post
- Paper
Multiplayer Stable Diffusion
There have been many creations of interactive 2D worlds where people can perform simple actions that combined together become a world on a grid. For instance, on yourworldoftext (beware it might not be moderated extensively) you can type text on a grid with other people, and in 2017 Reddit made an experiment where users could change the color of a single pixel on a grid each before having to wait and submit again.
Now a new experiment has been created involving Stable Diffusion! It’s a Hugging Face space you can access here, and here is a tweet from the creator talking about it. You simply choose the position of a box on the grid, type a prompt and then let it generate an image at that position. As of writing this, the grid seems to be invaded by Shrek…
Stable Diffusion Video
Finally, I would like to share this video that someone created with Stable Diffusion by looping and reapplying the model on parts of the image using an algorithm. Looks amazing:
That was it for this week, thanks for reading.
Check out the next article in the series:
If you’re interested in reading more articles about data science or AI, check out my reading lists below:
If you’d like to get a Medium membership you can use my referral link if you wish. Have a nice day.
