New Chip Design to Revolutionize AI in Portable Devices

 

The hardware bottleneck in AI and data science: the need for better memory capacity

As AI and data science applications continue to advance, the size of neural networks required to process data is doubling at a rapid pace. However, the hardware capability needed to support these networks is not keeping up with the demand. This has created a significant bottleneck in the field, as software is limited by the hardware on which it runs.

The problem is especially acute in portable devices, which require smaller and more energy-efficient chips to support heavy computation. While traditional silicon chips have been the primary hardware solution for decades, their limitations have become increasingly evident as the demands of AI and data science applications have grown.

The need for better memory capacity is a key driver of this bottleneck. Memory is crucial for storing and processing large amounts of data, and traditional silicon chips are limited in their ability to support high-density memory. As a result, researchers and industry experts are exploring new materials and devices to support better memory capacity and enable powerful AI in portable devices.

The development of new chip designs that combine traditional silicon technology with new materials is one promising approach to addressing the hardware bottleneck. By using metal oxide memristors to store memory, researchers are able to create powerful but low-energy intensive chips that can support heavy computation.

Furthermore, by focusing on using the positions of atoms to represent information rather than the number of electrons, the new chip designs offer a more compact and stable way to store information. This enables higher information density per device, which is crucial for supporting the demands of AI and data science applications.

Overall, the need for better memory capacity is a critical challenge facing the field of AI and data science. The development of new chip designs that combine traditional silicon technology with new materials offers a promising solution to this challenge, and could enable powerful AI in portable devices that is both energy-efficient and affordable.

Combining traditional silicon technology with new materials to support heavy computation

As the size of neural networks needed for AI and data science applications continues to double every 3.5 months, the hardware capability needed to process them has only been doubling every 3.5 years, creating a bottleneck. This challenge has prompted researchers to explore new materials and devices to address the problem, while others continue to work on hardware solutions using silicon chips.

One promising approach is to combine the advantages of new materials with traditional silicon technology to support heavy computation. USC Professor of Electrical and Computer Engineering Joshua Yang, along with his collaborators, has been exploring this approach in their research. They have developed a new type of chip with the best memory capacity of any chip thus far for edge AI (AI in portable devices) that combines silicon with metal oxide memristors to create powerful but low-energy intensive chips.

The team has developed a protocol for devices to reduce "noise" and demonstrated the practicality of using this protocol in integrated chips. This demonstration was made at TetraMem, a startup company co-founded by Yang and his co-authors to commercialize AI acceleration technology. This new memory chip has the highest information density per device among all known memory technologies thus far, making it a critical component in bringing incredible power to the devices in our pockets.

The chips are not just for memory but also for the processor, and millions of them in a small chip, working in parallel, could rapidly run AI tasks and require only a small battery to power it. The technique focuses on using the positions of atoms to represent information rather than the number of electrons, which is the current technique involved in computations on chips. The positions of the atoms offer a compact and stable way to store more information in an analog, instead of digital, fashion.

Moreover, the information can be processed where it is stored instead of being sent to one of the few dedicated 'processors,' eliminating the so-called 'von Neumann bottleneck' existing in current computing systems. In this way, computing for AI is more energy efficient with higher throughput.

By converting chips to rely on atoms instead of electrons, chips become smaller, and there is more computing capacity at a smaller scale. This method could offer many more levels of memory to help increase information density, making high-powered tech more affordable and accessible for all sorts of applications. The new innovation, followed by some further development, could put the power of advanced AI in everyone's personal device.

The breakthrough: using atoms instead of electrons to store memory

The breakthrough in the research by USC Professor of Electrical and Computer Engineering Joshua Yang and his collaborators is the use of atoms instead of electrons to store memory. Electrons are known to be "light" and volatile, making them prone to moving around and losing their stability. In contrast, atoms are much more stable and can store information in a compact and stable way. This allows for a much higher information density and eliminates the need for battery power to maintain stored information.

By storing memory in full atoms rather than manipulating electrons, the new chips developed by Yang and his colleagues can process information more efficiently and at a higher throughput. The positions of the atoms are used to represent information in an analog fashion, allowing for more computing capacity at a smaller scale. This method could offer many more levels of memory to help increase information density.

The new chips developed by Yang and his team combine traditional silicon technology with metal oxide memristors to create powerful but low-energy intensive chips. This technique focuses on using the positions of atoms to represent information rather than the number of electrons. By eliminating the need to send information to one of the few dedicated "processors," these chips can process information where it is stored, eliminating the von Neumann bottleneck that exists in current computing systems.

The potential of this breakthrough is significant. The new memory chip developed by Yang and his team has the highest information density per device (11 bits) among all known memory technologies so far. This high information density could play a critical role in bringing incredible power to the devices in our pockets, enabling powerful AI capabilities in edge devices such as Google Glasses. It could make such high-powered technology more affordable and accessible for all sorts of applications.

How the new chip design enables powerful AI in edge devices

The new chip design with the best memory of any chip thus far for edge AI could enable powerful AI in portable devices. By combining silicon with metal oxide memristors, the chip is both powerful and low-energy intensive. The information is stored in full atoms, which offer a compact and stable way to store more information in an analog fashion. This allows for high information density and eliminates the von Neumann bottleneck existing in current computing systems.

The chip can be used for both memory and the processor, allowing millions of them to work in parallel to rapidly run AI tasks with only a small battery to power it. This makes computing for AI more energy efficient with a higher throughput.

Furthermore, the stable memory capable of high information density is crucial for AI computations. The new chip design could enable powerful AI capability in edge devices such as Google Glasses, which previously suffered from frequent recharging issues. With the new chip, these devices could become more affordable and accessible for various applications, bringing incredible power to the devices in our pockets.

The potential impact of the innovation on affordability and accessibility of high-powered tech

The development of a new chip design with the best memory capacity for edge AI could have a significant impact on the affordability and accessibility of high-powered technology. Traditionally, powerful AI and data science computation requires expensive, large-scale hardware, which limits its accessibility to only those with the financial means to invest in it.

However, the new chip design could change this by enabling the integration of high-powered AI capabilities into small, portable devices such as smartphones, tablets, and wearables. By having access to AI in portable devices, people can benefit from the power of AI without having to invest in expensive hardware.

Furthermore, the new chip design's energy efficiency could also lead to cost savings in terms of battery usage, making these devices even more affordable and accessible to a broader range of people. This could have a significant impact on a variety of applications, from healthcare to education, where access to high-powered technology could enable new solutions and services.

Moreover, the development of the new chip design could lead to further innovation and competition, driving down costs and increasing accessibility even more. This could have a transformative impact on industries such as transportation, manufacturing, and agriculture, where AI and data science can revolutionize processes and improve efficiency.

In conclusion, the development of a new chip design with the best memory capacity for edge AI has the potential to democratize access to high-powered technology, making it more affordable and accessible to a broader range of people. The impact of this innovation could be transformative, with the potential to revolutionize industries and enable new solutions and services that were previously impossible.

Follow Like Susbcribe

https://thetechsavvysociety.com/
https://twitter.com/tomarvipul
https://thetechsavvysociety.blogspot.com/
https://www.instagram.com/thetechsavvysociety/
https://www.youtube.com/@thetechsavvysociety
https://medium.com/@tomarvipul
https://podcasts.apple.com/us/podcast/the-tech-savvy-society/id1675203399
https://open.spotify.com/show/10LEs6gMHIWKLXBJhEplqr

Journal Reference:

Mingyi Rao, Hao Tang, Jiangbin Wu, Wenhao Song, Max Zhang, Wenbo Yin, Ye Zhuo, Fatemeh Kiani, Benjamin Chen, Xiangqi Jiang, Hefei Liu, Hung-Yu Chen, Rivu Midya, Fan Ye, Hao Jiang, Zhongrui Wang, Mingche Wu, Miao Hu, Han Wang, Qiangfei Xia, Ning Ge, Ju Li, J. Joshua Yang. Thousands of conductance levels in memristors integrated on CMOS. Nature, 2023; 615 (7954): 823 DOI: 10.1038/s41586-023-05759-5