Top 10 research articles written by me

Summary

The website content is an author's reflection on their journey and top-cited research papers in the field of artificial intelligence and machine learning, spanning topics from Bayesian optimization to philosophical aspects of AI.

Abstract

The author celebrates reaching 500 citations by sharing a narrative that encapsulates their research trajectory and motivations since 2013. Initially inspired by the film Matrix, they embarked on a quest to understand the limits of artificial intelligence, delving into both technical and philosophical aspects. The author's work covers a range of topics, including the development of methods to handle categorical and integer variables in Bayesian optimization, comparisons of BERT with traditional machine learning for text classification, and the exploration of multi-objective Bayesian optimization with constraints. Their research also extends to the optimization of Bayesian networks, natural language summarization, Gaussian Process Classification with noisy inputs, and the theoretical boundaries of machine consciousness. The author reflects on their academic journey, the evolution of their interests, and the interplay between technical expertise and philosophical inquiry, inviting readers to engage with their work and contribute to the ongoing discourse.

Opinions

The author believes that technical expertise in AI also contributes to being a philosopher of the topic.
They express that the initial interest in the limits of AI was primarily a philosophical question, which they realized over time.
The author values the importance of empirical evidence, as demonstrated by their work comparing BERT to traditional machine learning algorithms.
They suggest that Bayesian networks are underrated by the machine learning community and see potential for further research.
The author is intrigued by the concept of machine consciousness and acknowledges the challenges in understanding phenomenal consciousness, while also enjoying it as a hobby.
They find joy in the application of Bayesian optimization to unconventional areas, such as suggesting cooking recipes.
The author considers it an honor to contribute to the DarkMachines project, highlighting the collaboration between physicists and machine learning practitioners.

Introduction and motivations

It has been a long journey, since 2013, when I wrote my first paper, now I have reached 500 citations according to Google Scholar. In this (meta-)article, I want to share a brief summary of my research and motivations.

I decided to study software engineering in 2008 driven by curiosity, and specially by the film Matrix, written by the Wachowskis. In particular, I was amazed by artificial intelligence, back when almost nobody cared about it from a business point of view, driven by the belief that it was mainly an academic topic. Concretely, I wondered about the limits of artificial intelligence, what was possible to simulate with AI? Intelligence? Consciousness? I believed that it was a technical issue. Looking back, thinking about the limits, that were my interests, was mainly a philosophical question. I discovered this afterwards, but being a technical expert also helps being a philosopher of the topic, so everything is OK and I have written lots of philosophical papers about AI in these years with my thoughts.

In my last year of undergraduate course, I wrote several papers on doing automatic summaries using classic AI. But then I discovered machine learning, that was huge, I believed! However, learning mathematics was mandatory, so I studied that by myself. After exploring the business world as a business consultant a pair of years, I went back to academia for my Msc and PhD, driven again by curiosity. And this is the result. Here I want to share a brief description of my main research papers.

My top 10 cited papers

Dealing with categorical and integer-valued variables in Bayesian optimization with Gaussian processes: It is great to perform Bayesian optimization with a GP as the model assumptions satisfy the main properties of the generalization error estimation of machine learning algorithms (like smoothness and continuity). However, not being able to model categorical variables (like tuning the activation function of NNs) or integer variables (like the number of layers of a NN) was a huge drawback. With a smart trick, an input transformation on the kernel of the GP, we can model these variables for a GP and perform GP-BO for hyper-parameter tuning of machine learning algorithms! The code is available at my Github page (https://github.com/EduardoGarrido90/).

Comparing BERT against traditional machine learning text classification: When I published this with the help of an undergraduate student it was not clear whether transformers outperform ML algorithms for supervised text classifications. Theoretically, it seemed, but more empirical evidence was needed. We performed several experiments in different platforms and languages to give enough empirical evidence about the superiority of transformers with respect to classical machine learning NLP.

Predictive Entropy Search for Multi-objective Bayesian Optimization with Constraints: The PES acquisition function perform a global estimation of the entropy lost by a new suggestion of Bayesian optimization. Intuitively, the point that maximizes the loss of entropy of the GP predictive distribution is chosen. Complex approximations must be done to the theoretical expression to estimate this loss of entropy. And even more complex approximations (with the expectation propagation algorithm) are needed to solve the constrained multi-objective scenario. The code is available at my Github page.

Bayesian optimization of the PC algorithm for learning Gaussian Bayesian networks: Bayesian networks, or probabilistic graphical models, are great, a really underestimated model by the ML community with a lot of applications. In this paper, we used Bayesian optimization with the PC algorithm to find better Bayesian network structures. A great line of research is open here, to develop an AutoBayesianNetwork package.

Creating a natural language summary from a compressed causal graph: My first paper that talks about my research when I was undergraduate. Basically we transformed a text into a causal graph and then we summarize it using a linear combination of several classical text dealing with semantics and syntax and AI algorithms to minimize redundancy and maximize relevance.

Multi-class Gaussian Process Classification with Noisy Inputs: A JMLR article that presents three methodologies to enhance GPs and deal with noise in the inputs. We use amortized inference and add new random variable to model this noise in sparse GPs. We show the usefulness of this methodology in an astrophysical problem that was the main motivation of these methods.

A Machine Consciousness Architecture Based on Deep Learning and Gaussian Processes: When I first explored the limits of AI, I was a functionalist, convinced that consciousness is explained by its “function”. Soon I started studying philosophy of mind and realized that I was accepting a lot of assumptions whose answer lies far from the scientific frontier of knowledge. However, we can simulate things as imagination, motion, access consciousness and more. We can for example implement a simulation of the GWT architecture. The hard problem of consciousness remains epistemologically unreachable and, although the IIT of Tononi tries to quantify phenomenal consciousness and models it through the qualia space and the FEP tries to give an explanation, in my opinion, we are not ready yet to understand phenomenal consciousness. You can visit my academia profile (https://upco.academia.edu/EduardoC%C3%A9sarGarridoMerch%C3%A1n) for more research on this, it is like a hobby for me.

Suggesting Cooking Recipes Through Simulation and Bayesian Optimization: Funny article where a student of mine made several salads and hot dogs whose recipe was tuned with Bayesian optimization according to the preferences of several people that daily tested the recipes. Google made a similar experiment with cookies.

A comparison of optimisation algorithms for high-dimensional particle and astrophysics applications: DarkMachines is a platform where physicians and ML practitioners test several ML algorithms or methodologies in different astrophysic problems. It was an honour to participate in these experiments.

Do you want to know more of a particular line of research of the ones cited here? Please leave your opinion in the comments!! See you soon and do not forget to have fun!