Amazing Science

National Supercomputing Center in Wuxi builds a yet another supercomputer that claims exascale performance. After the U.S. government imposed crippling sanctions against select Chinese high-tech and supercomputer companies through 2019 and 2020, firms like Huawei had to halt chip development; it is impossible to build competitive processors without access to leading-edge nodes. But Jiangnan Computing Lab, which develops Sunway processors, and National Supercomputing Center in Wuxi kept building new supercomputers and recently even submitted results of their latest machine for the Association for Computing Machinery's Gordon Bell prize.

The new Sunway supercomputer built by the National Supercomputing Center in Wuxi (an entity blacklisted in the U.S.) employs around feature approximately 19.2 million cores across 49,230 nodes, reports Supercomputing.org. To put the number into context, Frontier, the world's highest-performing supercomputer, uses 9472 nodes and consumes 21 MW of power. Meanwhile, the National Supercomputing Center in Wuxi does not disclose power consumption of its latest system.

Interestingly, the new supercomputer seems to be based on the already known 390-core Sunway processor that derive from the Sunway SW26010 CPUs and have been around since 2021. Therefore, the new system increased the number of processors, but not their architectural efficiency, so its power consumption is likely to be gargantuan. Meanwhile, actual performance of the machine is unknown, since scaling out has its limits even in the supercomputer world.

The National Supercomputing Center in Wuxi has not disclosed performance numbers of its new supercomputer, and it is hard to make any estimations about its performance at this point. The reason why we called it ‘exascale’ is because its predecessor, the Sunway Oceanlite from 2021, was estimated to offer compute performance of around 1 ExaFLOPS.

Meanwhile, engineers revealed the workload that it used the machine for. Apparently, the the group created a new code for large whirlpool simulations to address compressible currents in turbo machinery. They applied it to NASA’s grand challenge problem using an advanced unstructured solver for a high-pressure turbine sequence with 1.69 billion mesh components and 865 billion degrees of freedom (judged by the number of variables). Given how complex the simulation is, it is likely that the machine is indeed quite powerful. Meanwhile, there is no word whether the simulation was conducted with FP64 precision, or precision was sacrificed for the sake of performance.

OpenAI today announced the general availability of GPT-4, its latest text-generating model, through its API. Starting this afternoon, all existing OpenAI API developers “with a history of successful payments” can access GPT-4. The company plans to open up access to new developers by the end of this month, and then start raising availability limits after that “depending on compute availability.

"Millions of developers have requested access to the GPT-4 API since March, and the range of innovative products leveraging GPT-4 is growing every day,” OpenAI wrote in a blog post. “We envision a future where chat-based models can support any use case.” GPT-4 can generate text (including code) and accept image and text inputs — an improvement over GPT-3.5, its predecessor, which only accepted text — and performs at “human level” on various professional and academic benchmarks. Like previous GPT models from OpenAI, GPT-4 was trained using publicly available data, including from public webpages, as well as data that OpenAI licensed. The image understanding capability isn’t available to all OpenAI customers just yet. OpenAI’s testing it with a single partner, Be My Eyes, to start with. But it hasn’t indicated when it’ll open it up to the wider customer base.

It’s worth noting that, as with even the best generative AI models today, GPT-4 isn’t perfect. It “hallucinates” facts and makes reasoning errors, sometimes with confidence. And it doesn’t learn from its experience, failing at hard problems such as introducing security vulnerabilities into code it generates. In the future, OpenAI says that it’ll allow developers to fine-tune GPT-3 and GPT-3.5 Turbo, one of its other recent but less capable text-generating models, with their own data, as has long been possible with OpenAI’s other text-generating models. That capability should arrive later this year, according to OpenAI.

GPT-3.5 Turbo, DALL·E and Whisper APIs are also generally available, and we are releasing a deprecation plan for older models of the Completions API, which will retire at the beginning of 2024.

A new technique greatly reduces the error in an optical neural network, which uses light to process data instead of electrical signals. With their technique, the larger an optical neural network becomes, the lower the error in its computations. This could enable them to scale these devices up so they would be large enough for commercial uses.

As machine-learning models become larger and more complex, they require faster and more energy-efficient hardware to perform computations. Conventional digital computers are struggling to keep up.

An analog optical neural network could perform the same tasks as a digital one, such as image classification or speech recognition, but because computations are performed using light instead of electrical signals, optical neural networks can run many times faster while consuming less energy.

However, these analog devices are prone to hardware errors that can make computations less precise. Microscopic imperfections in hardware components are one cause of these errors. In an optical neural network that has many connected components, errors can quickly accumulate.

Even with error-correction techniques, due to fundamental properties of the devices that make up an optical neural network, some amount of error is unavoidable. A network that is large enough to be implemented in the real world would be far too imprecise to be effective.

MIT researchers have overcome this hurdle and found a way to effectively scale an optical neural network. By adding a tiny hardware component to the optical switches that form the network’s architecture, they can reduce even the uncorrectable errors that would otherwise accumulate in the device.

Their work could enable a super-fast, energy-efficient, analog neural network that can function with the same accuracy as a digital one. With this technique, as an optical circuit becomes larger, the amount of error in its computations actually decreases.  

“This is remarkable, as it runs counter to the intuition of analog systems, where larger circuits are supposed to have higher errors, so that errors set a limit on scalability. This present paper allows us to address the scalability question of these systems with an unambiguous ‘yes,’” says lead author Ryan Hamerly, a visiting scientist in the MIT Research Laboratory for Electronics (RLE) and Quantum Photonics Laboratory and senior scientist at NTT Research.

Hamerly’s co-authors are graduate student Saumil Bandyopadhyay and senior author Dirk Englund, an associate professor in the MIT Department of Electrical Engineering and Computer Science (EECS), leader of the Quantum Photonics Laboratory, and member of the RLE. The research is published today in Nature Communications.

Moore’s Law predicts that the number of transistors per device will double every two years. Moore’s Law is and always has been driven by innovation. The above figure illustrates the number of transistors per device as we look to the past, the present and the future. For the first 40 years, the gains came primarily from innovations in our process. Going forward, gains will come from innovations in both process and packaging. INTEL's processes will continue to deliver historic density improvements, while its 2D and 3D stacking technologies give architects and designers more tools to increase the number of transistors per device. As the designers look forward to innovative technologies such as High NA, RibbonFET, PowerVia, Foveros Omni and Direct, and others, INTEL sees no end to innovation and therefore currently no end to Moore’s Law.

In summary, when we consider all the various process and advanced packaging innovations, there are numerous options available to continue to double the number of transistors per device at the cadence demanded by our customers. Moore’s Law only stops when innovation stops, and innovation continues unabated at Intel in process, packaging and architecture. We remain undeterred in our aspiration to deliver approximately 1 trillion transistors in a single device by 2030.

Short animations giving viewers a taste of the tactics behind misinformation can help to “inoculate” people against harmful content on social media when deployed in YouTube’s advert slot, according to a major online experiment led by the University of Cambridge.

Micron Technology says it has reached volume production of a 232-layer NAND flash-memory chip. It’s the first such chip to pass the 200-layer mark, and it’s been a tight race. Competitors are currently providing 176-layer technology, and some have said they are on track to follow Micron’s skyward move or already have working chips in hand.

The new Micron tech as much as doubles the density of bits stored per unit area versus competing chips, packing in 14.6 gigabits per square millimeter. Its 1-terabit chips are bundled into 2-terabyte packages, each of which is barely more than a centimeter on a side and can store about two weeks worth of 4K video. With 81 trillion gigabytes (81 zettabytes) of data generated in 2021 and International Data Corp. (IDC) predicting 221 ZB in 2026, “storage has to innovate to keep up,” says Alvaro Toledo, Micron’s vice president of data-center storage.

The move to 223 layers is a combination and extension of many technologies Micron has already deployed. To get a handle on them, you need to know the basic structure and function of 3D NAND flash. The chip itself is made up of a bottom layer of CMOS logic and other circuitry that’s responsible for controlling reading and writing operations and getting data on and off the chip as quickly and efficiently as possible. Improvements to this layer, such as optimizing the path data travels and reducing the capacitance of the chip’s inputs and outputs, yielded a 50 percent improvement in the data transfer rate to 2.4 Gb/s.

Above the CMOS are layers upon layers of NAND flash cells. Unlike other devices, Flash-memory cells are built vertically. They start as a (relatively) deep, narrow hole etched through alternating layers of conductor and insulator. Then the holes are filled with material and processed to form the bit-storing part of the device. It’s the ability to reliably etch and fill the holes through all those layers that’s a key limit to the technology. Instead of etching through all 232 layers in one go, Micron’s process builds them in two parts and stacks one atop the other. Even so, “it’s an astounding engineering feat,” says Alvaro. “That was one of the biggest challenges we overcame.”

According to Toledo, there is a path toward even more layers in future NAND chips. “There are definitely challenges,” he says. But “we haven’t seen the end of that path.” In addition to adding more and more layers, NAND flash makers have been increasing the density of stored bits by packing multiple bits into a single device. Each of the Micron chip’s memory cells is capable of storing three bits per cell. That is, the charge stored in each cell produces a distinct enough effect to discern eight different states. Though 3-bit-per-cell products (called TLC) are the majority, four-bit products (called QLC) are also available. One QLC chip presented by Western Digital researchers at the IEEE International Solid State Circuits Conference earlier this year achieved a 15 Gb/mm2 areal density in a 162-layer chip. And Kioxia engineers reported 5-bit cells last month at the IEEE Symposium on VLSI Technology and Circuits. There has even been a 7-bit cell demonstrated, but it required dunking the chip in 77-kelvin liquid nitrogen.

Medical schools are using augmented reality in their curricula to give students hands-on learning opportunities. Medical students can use augmented reality to see and apply theories throughout their training.

Neil deGrasse Tyson once said that we are either living in a simulation or on the verge of creating a simulation that others will live in. Imagine we become the agents of that VR into a world like that and interact with AI who might one day realize they aren't real and try to escape into our world.

Labeling data can be a chore. It's the main source of sustenance for computer-vision models; without it, they'd have a lot of difficulty identifying objects, people, and other important image characteristics. Yet producing just an hour of tagged and labeled data can take a whopping 800 hours of human time. Our high-fidelity understanding of the world develops as machines can better perceive and interact with our surroundings. But they need more help.

Scientists from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL), Microsoft, and Cornell University have attempted to solve this problem plaguing vision models by creating "STEGO," an algorithm that can jointly discover and segment objects without any human labels at all, down to the pixel.

STEGO learns something called "semantic segmentation"—fancy speak for the process of assigning a label to every pixel in an image. Semantic segmentation is an important skill for today's computer-vision systems because images can be cluttered with objects. Even more challenging is that these objects don't always fit into literal boxes; algorithms tend to work better for discrete "things" like people and cars as opposed to "stuff" like vegetation, sky, and mashed potatoes. A previous system might simply perceive a nuanced scene of a dog playing in the park as just a dog, but by assigning every pixel of the image a label, STEGO can break the image into its main ingredients: a dog, sky, grass, and its owner.

Assigning every single pixel of the world a label is ambitious—especially without any kind of feedback from humans. The majority of algorithms today get their knowledge from mounds of labeled data, which can take painstaking human-hours to source. Just imagine the excitement of labeling every pixel of 100,000 images. To discover these objects without a human's helpful guidance, STEGO looks for similar objects that appear throughout a dataset. It then associates these similar objects together to construct a consistent view of the world across all of the images it learns from.

Seeing the world

Machines that can "see" are crucial for a wide array of new and emerging technologies like self-driving cars and predictive modeling for medical diagnostics. Since STEGO can learn without labels, it can detect objects in many different domains, even those that humans don't yet understand fully.

"If you're looking at oncological scans, the surface of planets, or high-resolution biological images, it's hard to know what objects to look for without expert knowledge. In emerging domains, sometimes even human experts don't know what the right objects should be," says Mark Hamilton, a Ph.D. student in electrical engineering and computer science at MIT, research affiliate of MIT CSAIL, software engineer at Microsoft, and lead author on a new paper about STEGO. "In these types of situations where you want to design a method to operate at the boundaries of science, you can't rely on humans to figure it out before machines do."

Immunohistochemical stains for individual markers revolutionized diagnostic pathology decades ago but cannot capture enough information to accurately predict response to immunotherapy. Newer multiplex immunofluorescent technologies provide the potential to visualize the expression patterns of many functionally relevant molecules but present numerous challenges in accurate image analysis and data handling, particularly over large tumor areas. Drawing from the field of astronomy, in which petabytes of imaging data are routinely analyzed across a wide spectral range, Berry et al. developed a platform for multispectral imaging of whole-tumor sections with high-fidelity single-cell resolution. The resultant AstroPath platform was used to develop a multiplex immunofluorescent assay highly predictive of responses and outcomes for melanoma patients receiving immunotherapy.

The process of reconstructing experiences from human brain activity offers a unique lens into how the brain interprets and represents the world. Recently, the Google team and international collaborators introduced a method for reconstructing music from brain activity alone, captured using functional magnetic resonance imaging (fMRI). This approach uses either music retrieval or the MusicLM music generation model conditioned on embeddings derived from fMRI data. The generated music resembles the musical stimuli that human subjects experienced, with respect to semantic properties like genre, instrumentation, and mood. The scientists investigate the relationship between different components of MusicLM and brain activity through a voxel-wise encoding modeling analysis. Furthermore, they analyze which brain regions represent information derived from purely textual descriptions of music stimuli.

Combining Wolfram Alpha and GPT-3 could potentially create a powerful tool that could answer complex questions and generate human-like text. It could be used to provide in-depth explanations and discussions on a wide range of topics, and could be used to engage in natural-sounding conversations.

Accessing Wolfram|Alpha's computational knowledge with ChatGPT--an ideal combination of precise computation with human-like expression of ideas. Stephen Wolfram explains how.

It’s always amazing when things suddenly “just work”. It happened to us with Wolfram|Alpha back in 2009. It happened with our Physics Project in 2020. And it’s happening now with OpenAI’s ChatGPT. Stephen Wolfram has been tracking neural net technology for about 43 years and he finds the performance of ChatGPT thoroughly remarkable. Suddenly, there is a system that can successfully generate text about almost anything—that’s very comparable to what humans might write. It’s impressive, and useful and its success is probably going to tell us some very fundamental things about the nature of human thinking.

But while ChatGPT is a remarkable achievement in automating the doing of major human-like things, not everything that’s useful to do is quite so “human like”. Some of it is instead more formal and structured. And indeed one of the great achievements of our civilization over the past several centuries has been to build up the paradigms of mathematics, the exact sciences—and, most importantly, now computation—and to create a tower of capabilities quite different from what pure human-like thinking can achieve.

Stephen Wolfram himself has been deeply involved with the computational paradigm for many decades, in the singular pursuit of building a computational language to represent as many things in the world as possible in formal symbolic ways. His goal has always been to build a system that can “computationally assist”—and augment—what I and others want to do. He thinks about things as a human but can also immediately use Wolfram Language and Wolfram|Alpha to tap into a kind of unique “computational superpower” that lets him perform all sorts of beyond-human things. It’s a tremendously powerful way of working and not just important for humans but equally, if not more, important for human-like AIs as well—immediately giving them computational knowledge superpowers, that leverage the non-human-like power of structured computation and structured knowledge.

We’ve just started exploring what this means for ChatGPT. But it’s pretty clear that wonderful things are now possible. Wolfram|Alpha does something very different from ChatGPT, in a very different way. But they have a common interface: natural language. And this means that ChatGPT can “talk to” Wolfram|Alpha just like humans do—with Wolfram|Alpha turning the natural language it gets from ChatGPT into precise, symbolic computational language on which it can apply its computational knowledge power.

For decades there’s been a dichotomy in thinking about AI between “statistical approaches” of the kind ChatGPT uses, and “symbolic approaches” that are in effect the starting point for Wolfram|Alpha. But now—thanks to the success of ChatGPT—as well as all the work we’ve done in making Wolfram|Alpha understand natural language—there’s finally the opportunity to combine these to make something much stronger than either could ever achieve on their own.

A new AI program observed physical phenomena and uncovered relevant variables—a necessary precursor to any physics theory.

All physical laws are described as mathematical relationships between state variables. These variables give a complete and non-redundant description of the relevant system. However, despite the prevalence of computing power and artificial intelligence, the process of identifying the hidden state variables themselves has resisted automation. Most data-driven methods for modeling physical phenomena still rely on the assumption that the relevant state variables are already known. A longstanding question is whether it is possible to identify state variables from only high-dimensional observational data.

Scientists now created a principle for determining how many state variables an observed system is likely to have, and what these variables might be. They were able to demonstrate the effectiveness of this approach using video recordings of a variety of physical dynamical systems, ranging from elastic double pendulums to fire flames. Without any prior knowledge of the underlying physics, our algorithm discovers the intrinsic dimension of the observed dynamics and identifies candidate sets of state variables.

Github repository is here

A new database and searchable tool reveals more than 90,000 known materials with electronic properties that remain unperturbed in the face of disruption.

Topology stems from a branch of mathematics that studies shapes that can be manipulated or deformed without losing certain core properties. A donut is a common example: If it were made of rubber, a donut could be twisted and squeezed into a completely new shape, such as a coffee mug, while retaining a key trait — namely, its center hole, which takes the form of the cup’s handle. The hole, in this case, is a topological trait, robust against certain deformations.

In recent years, scientists have applied concepts of topology to the discovery of materials with similarly robust electronic properties. In 2007, researchers predicted the first electronic topological insulators — materials in which electrons that behave in ways that are “topologically protected,” or persistent in the face of certain disruptions.

Since then, scientists have searched for more topological materials with the aim of building better, more robust electronic devices. Until recently, only a handful of such materials were identified, and were therefore assumed to be a rarity. Now researchers at MIT and elsewhere have discovered that, in fact, topological materials are everywhere, if you know how to look for them.

In a paper published today in Science, the team, led by Nicolas Regnault of Princeton University and the École Normale Supérieure Paris, reports harnessing the power of multiple supercomputers to map the electronic structure of more than 96,000 natural and synthetic crystalline materials. They applied sophisticated filters to determine whether and what kind of topological traits exist in each structure.

Overall, they found that 90 percent of all known crystalline structures contain at least one topological property, and more than 50 percent of all naturally occurring materials exhibit some sort of topological behavior. “We found there’s a ubiquity — topology is everywhere,” says Benjamin Wieder, the study’s co-lead, and a postdoc in MIT’s Department of Physics.

The team has compiled the newly identified materials into a new, freely accessible Topological Materials Database resembling a periodic table of topology. With this new library, scientists can quickly search materials of interest for any topological properties they might hold, and harness them to build ultra-low-power transistors, new magnetic memory storage, and other devices with robust electronic properties.

Synthetically generated faces are not just highly photorealistic, they are nearly indistinguishable from real faces…. Perhaps most interestingly, we find that synthetically generated faces are more trustworthy than real faces. This may be because synthesized faces tend to look more like average faces which themselves are deemed more trustworthy. 

Regardless of the underlying reason, synthetically generated faces have emerged on the other side of the uncanny valley. This should be considered a success for the fields of computer graphics and vision. At the same time, easy access to such high-quality fake imagery has led and will continue to lead to various problems, including more convincing online fake profiles and—as synthetic audio and video generation continues to improve—problems of nonconsensual intimate imagery, fraud, and disinformation campaigns, with serious implications for individuals, societies, and democracies.

A team of physicists from the Harvard-MIT Center for Ultracold Atoms and other universities has developed a special type of quantum computer known as a programmable quantum simulator capable of operating with 256 quantum bits, or "qubits."

The system marks a major step toward building large-scale quantum machines that could be used to shed light on a host of complex quantum processes and eventually help bring about real-world breakthroughs in material science, communication technologies, finance, and many other fields, overcoming research hurdles that are beyond the capabilities of even the fastest supercomputers today. Qubits are the fundamental building blocks on which quantum computers run and the source of their massive processing power.

"This moves the field into a new domain where no one has ever been to thus far," said Mikhail Lukin, the George Vasmer Leverett Professor of Physics, co-director of the Harvard Quantum Initiative, and one of the senior authors of the study published today in the journal Nature. "We are entering a completely new part of the quantum world." According to Sepehr Ebadi, a physics student in the Graduate School of Arts and Sciences and the study's lead author, it is the combination of system's unprecedented size and programmability that puts it at the cutting edge of the race for a quantum computer, which harnesses the mysterious properties of matter at extremely small scales to greatly advance processing power. Under the right circumstances, the increase in qubits means the system can store and process exponentially more information than the classical bits on which standard computers run.

"The number of quantum states that are possible with only 256 qubits exceeds the number of atoms in the solar system," Ebadi said, explaining the system's vast size. Already, the simulator has allowed researchers to observe several exotic quantum states of matter that had never before been realized experimentally, and to perform a quantum phase transition study so precise that it serves as the textbook example of how magnetism works at the quantum level.

These experiments provide powerful insights on the quantum physics underlying material properties and can help show scientists how to design new materials with exotic properties. The project uses a significantly upgraded version of a platform the researchers developed in 2017, which was capable of reaching a size of 51 qubits. That older system allowed the researchers to capture ultra-cold rubidium atoms and arrange them in a specific order using a one-dimensional array of individually focused laser beams called optical tweezers.

This new system allows the atoms to be assembled in two-dimensional arrays of optical tweezers. This increases the achievable system size from 51 to 256 qubits. Using the tweezers, researchers can arrange the atoms in defect-free patterns and create programmable shapes like square, honeycomb, or triangular lattices to engineer different interactions between the qubits.