These 9 Research Papers are changing how I see Artificial Intelligence this year.

A list of Research papers can be a tool that helps you to foresee the future of AI

Artificial intelligence research has come a long way since its first steps in teaching computer board game strategies in the 1950s.

Machine learning, and its promise of real-time algorithm improvements through experience and access to more data in the twenty-first century, has become the most important research focus in the field.

In a global picture, the United States continues to lead the world in AI research papers published in 2021, but China is on track to overtake traditional artificial intelligence research powerhouses in the coming years, thanks to an astronomical increase over the last two decades.

To give you an idea, the Eastern Asian country surpassed the number of AI research papers published in all 27 EU countries combined in 2008. It now ranks second, with approximately 138.000 papers scheduled for publication in 2021. Over the last 20 years, it has increased its research output by 3,350%.

Every year, scientists worldwide publish thousands of research papers on AI. Still, only a few reach a large audience and have a global impact.

I am very passionate about reading, understanding, and experimenting (when possible) with research papers related to machine learning and deep learning; I believe it helps me to keep updated in my field, and it helps me to “see the future” since the advancements we benefit today from Artificial Intelligence were brought to us thanks to these kinds of research.

In this article, I will share some of the most interesting papers I’ve been reading this year that I believe are impacting how I see Artificial Intelligence today and soon.

The list is not based on the number of citations or their inclusion in major AI conferences and journals, but it is a personal preference based on my daily interaction with Artificial Intelligence.

I’ll do my best to explain why you should care about each of these papers and where they fit into the big picture.

I hope this list can also inspire you to use research papers as a tool to foresee the future of AI.

1. Open Pre-trained Transformers (OPT)

Facebook (ops.. I should get used to the idea of calling it Meta) has just released OPT, a 175 billion parameter language model…

Large language models, often trained over hundreds of thousands of computing days, have demonstrated remarkable zero- and few-shot learning capabilities.

Due to their computational cost, these models are hard to replicate without significant capital.

The few available APIs do not allow access to the full model weights, making them difficult to study.

Facebook introduced Open Pre-trained Transformers (OPT), a collection of decoder-only pre-trained transformers with parameters ranging from 125M to 175B that is freely and responsibly shared with interested researchers.

According to Facebook researchers, OPT-175B is comparable to GPT-3 and requires only one-seventh of the carbon footprint to develop.

They are also making available their logbook, which details their infrastructure challenges, and code for experimenting with all the released models.

2. Resonance as a Design Strategy for AI and Social Robots

The powerful and pervasive resonance phenomenon plays a significant role in human interactions.

This article investigates the relationship between the physical mechanism of resonance and the human experience of resonance and potential ways to improve the experience of resonance in human-robot interactions.

The authors begin by introducing resonance as a popular cultural and scientific metaphor. They then discuss the physical mechanism of “sympathetic resonance.”

The rest of the article is divided into two sections following this introduction. Part one examines the role of resonance in human cognition and social interactions (including synchronization and rhythmic entrainment).

Part two examines resonance-related phenomena in robotics and artificial intelligence (AI).

These two reviews are the foundation for presenting a design strategy and combinatorial design space for shaping resonant interactions with robots and AI.

They end by posing hypotheses and research questions for future empirical studies and discussing various ethical and aesthetic issues related to resonance in human-robot interactions.

3. Using machine learning to understand age and gender classification based on infant temperament

Age and gender differences are prominent in temperament literature, with the former particularly salient in infancy and the latter as early as the first year of life.

This study represents a meta-analysis utilizing Infant Behavior Questionnaire-Revised (IBQ-R) data collected across multiple laboratories to overcome the limitations of smaller samples in elucidating links between temperament, age, and gender in early childhood.

Algorithmic modeling techniques were leveraged to discern how much the 14 IBQ-R subscale scores accurately classified participating children as boys and girls and into three age groups: youngest, mid-range, and oldest.

Additionally, simultaneous classification into age and gender categories was performed, allowing an opportunity to consider how much gender differences in temperament are informed by infant age.

Results indicated that age group classification was more accurate than child gender models, suggesting that age-related changes are more salient than gender differences in early childhood concerning temperament attributes.

However, gender-based classification was superior in the oldest age group, suggesting temperament differences between boys and girls are accentuated with development. Fear emerged as the subscale contributing to accurate classifications, most notably overall.

This study leads infancy research and meta-analytic investigations more broadly in a new direction as a methodological demonstration. Also, it provides the most optimal comparative data for the IBQ-R based on the largest and most representative dataset to date.

4. READ: Large-Scale Neural Scene Rendering for Autonomous Driving

This is a research paper about a large-scale neural rendering method proposed to synthesize the autonomous driving scene (READ), allowing large-scale driving scenarios to be synthesized on a PC.

The deep learning model can not only create realistic driving scenes but can also stitch and edit them.

Free-view photorealistic image synthesis is a critical task in multimedia.

Experimenting with different scenarios has become a challenge with the development of advanced driver assistance systems (ADAS) and their applications in autonomous vehicles.

Although image-to-image translation methods can produce photorealistic street scenes, they cannot make coherent scenes due to a lack of 3D information.

This paper proposes a large-scale neural rendering method to synthesize the autonomous driving scene (READ), allowing large-scale driving scenarios to be synthesized on a PC using various sampling schemes.

The authors propose a net rendering network to learn neural descriptors from sparse point clouds to represent driving scenarios.

Experiments show that the model is effective in large-scale driving scenarios.

READ: Large-Scale Neural Scene Rendering for Autonomous Driving

Synthesizing free-view photo-realistic images is an important task in multimedia. With the development of advanced…

arxiv.org

5. A scholarly network of AI research with an information science focus: Global North and Global South perspectives

This paper’s primary goal is to provide a citation-based method for exploring the scholarly network of artificial intelligence (AI)-related research in the information science (IS) domain, particularly from the perspectives of the Global North (GN) and the Global South (GS).

Three research objectives were addressed:

the field’s publication patterns
The most influential articles and keywords researched in the field.
The scholarly network of GN and GS researchers between 2010 and 2020 is visualized.

The PRISMA statement was used to retrieve and analyze longitudinal research data from the Web of Science. To find relevant quality articles, 32 AI-related keywords were used. Finally, 149 articles accompanying the follow-up 8838 citing articles were identified as eligible sources. A co-citation network analysis was used to scientifically visualize the intellectual structure of AI research in GN and GS networks.

According to the findings, the most productive GN countries are the United States, Australia, and the United Kingdom. In contrast, the most productive GS countries are China and India.

The top ten co-cited AI research articles in the IS domain were then identified. Third, the scholarly AI research networks in the GN and GS areas were visualized. GN researchers in the IS domain focused on applied research involving intelligent systems (e.g., decision support systems) between 2010 and 2015; GS researchers focused on big data applications between 2016 and 2020. (e.g., geospatial big data research).

Throughout the study period, both GN and GS researchers concentrated on technology adoption research (e.g., AI-related products and services). Overall, the intellectual structure of the scholarly network on AI research and several applications in the IS literature is revealed in this paper.

The findings provide research-based evidence for further development.

6. Simultaneous emulation of synaptic and intrinsic plasticity using a memristive synapse.

Mariana Verzaro, a Ph.D. researcher in Neuroscience, inspired me to write about neuromorphic computing. I believe this area will garner a lot of attention in the coming decade, owing to AI, ML, and DL’s ability to elaborate on large amounts of data and identify patterns.

The hardware embodiment of neural networks is the focus of neuromorphic computing, and device implementation of individual neurons and synapses has received much attention.

Since the advent of memristors, the emulation of synaptic plasticity has yielded promising results.

However, intrinsic neuronal plasticity has rarely been demonstrated, which involves the learning process via synaptic plasticity interactions.

Synaptic and intrinsic plasticity occur concurrently during the learning process, implying the need for concurrent implementation.

This academic paper, titled “Simultaneous emulation of synaptic and intrinsic plasticity using a memristive synapse” and published in May 2022 by researchers at Korea Advanced Institute of Science and Technology, describes a neurosynaptic device that mimics synaptic and intrinsic plasticity simultaneously in a single cell.

The threshold switch-phase memory device combines a threshold switch and a phase change memory.

The bottom threshold switch layer, which resembles the modulation of firing frequency in the biological neuron, is used to demonstrate intrinsic neuronal plasticity.

The top phase change layer’s nonvolatile switching also contributes to synaptic plasticity. Intrinsic and synaptic plasticity are emulated in a single cell to establish positive feedback.

A positive feedback learning loop mimics the retraining process of a biological system. It is implemented in a threshold switch-phase change memory array for accelerated training.

7. Social Interpretable Tree for Pedestrian Trajectory Prediction

Understanding the various socially acceptable future behaviors is critical for many vision applications, including the navigation of autonomous vehicles.

So this week’s Sunday reading is a paper that proposes a tree-based method called Social Interpretable Tree (SIT) to address multi-modal prediction tasks, in which a hand-crafted tree is built based on preliminary information from an observed trajectory to model multiple future trajectories.

For example, a path in the tree from the root to the leaf represents an individual’s potential future trajectory.

SIT uses a coarse-to-fine optimization strategy. The tree is first built with high-order velocity to balance its complexity and coverage and then greedily optimized to encourage multimodality.

Finally, the final fine trajectory is predicted using a teacher-forcing refining operation.

In contrast to previous methods that used implicit latent variables to represent possible future trajectories, the path in the tree can explicitly explain the rough moving behaviors (e.g., go straight and turn right), providing better interpretability.

Despite the hand-crafted tree, experimental results on ETH-UCY and Stanford Drone datasets show that this method can match or outperform state-of-the-art techniques.

These experiments show that the untrained built tree outperforms many prior deep neural network-based approaches. Meanwhile, the method provides adequate flexibility in long-term prediction and various best-of-best predictions.

8. Energy-Efficient Parking Analytics System using Deep Reinforcement Learning

This paper discusses how advances in deep vision techniques and the widespread availability of smart cameras have fueled the next generation of video analytics.

However, video analytics applications consume enormous energy due to the power requirements of both deep learning techniques and cameras.

The author focuses on a parking video analytics platform in this paper and proposes RL-CamSleep, a deep reinforcement learning-based technique, to actuate the cameras to reduce the energy footprint while retaining the system’s utility.

The key insight is that many video-analytics applications do not always need to be operational. We can design policies to activate video analytics only when required.

Furthermore, the work complements existing work on improving hardware and software efficiency.

The paper evaluates the approach on a city-scale parking dataset with 76 streets spread across the city.

The analysis shows how streets have different parking patterns, emphasizing the importance of an adaptive policy.

In video analytics, the approach can learn an adaptive policy that can reduce average energy consumption by 76.38 percent while achieving an average accuracy of more than 98 percent.

9. Word2vec: Using Language Models to Understand Ware Premia

Using this AI model trained on over one million online job postings, you can evaluate the salary-relevant characteristics of jobs in near-real time.

The author obtained more than one million pre-pandemic job postings from Greenwich. In addition, HR aggregates millions of job postings from online job board platforms to analyze how the text of online job postings relates to salaries.

It turns out that we can use job listing text to evaluate salary-relevant job characteristics in near real-time.

This information could improve the transparency of job applications and our approach to workforce education and training.

The AI is based on BERT, one of the most advanced natural language processing (NLP) models, trained on over 800,000 job postings and salary data.

When tested with the remaining 200,000 job listings, it correctly predicted the associated salaries 87% of the time.

In comparison, using only the job titles and geographic locations from the job postings produced accurate predictions only 69% of the time.

Conclusion

That’s it for now. However, significant ethical and technical challenges must be addressed as artificial intelligence (AI) becomes more prevalent in medicine, education, and security.

If you have questions about the papers, please leave a comment or contact me.

You can also subscribe to my blog to receive notifications whenever I post on Medium.

My books about Artificial Intelligence

Also, I’ve just published other interesting ebooks on Amazon, and I’m sure some of them may be interesting for you… let’s keep in touch, follow me and do it together.

A.I. in 2020: A Year writing about Artificial Intelligence

AI, Robotics and Coding (for Parents): A practical guide for analog parents with digital kids

The Terminator paradox: How neuroscience can help us to understand Empathy and the fear of Artificial Intelligence

Artificial Intelligence from A to Z: Demystifying the essential concepts of AI

Sources, links, and references

Would you like to support me?

To get access to unlimited stories, you can also consider signing up to become a Medium member for just $5. In addition, if you sign up using my link, I’ll receive a small commission (at no extra cost to you).

Join Medium with my referral link - Jair Ribeiro

Read every story I share (and thousands of other writers on Medium). Your membership fee directly supports other…

jairribeiro.medium.com