Nature Climate Change, Published online: 22 July 2025; doi:10.1038/s41558-025-02393-z

Sublethal impacts of heat on reproductive outcomes are beginning to be considered as important drivers of population persistence under climate change. Now, research shows that the impact of transient heat on antipredator behaviours may be an underappreciated source of variation that could have far-reaching implications for survival.

The federal government is trying to use Medicaid data to identify and deport immigrants. So EFF and our friends at EPIC and the Protect Democracy Project have filed an amicus brief asking a judge to block this dangerous violation of federal data privacy laws.

Last month, the AP reported that the U.S. Department of Health and Human Services (HHS) had disclosed to the U.S. Department of Homeland Security (DHS) a vast trove of sensitive data obtained from states about people who obtain government-assisted health care. Medicaid is a federal program that funds health insurance for low-income people; it is partially funded and primarily managed by states. Some states, using their own funds, allow enrollment by non-citizens. HHS reportedly disclosed to DHS the Medicaid enrollee data from several of these states, including enrollee names, addresses, immigration status, and claims for health coverage.

In response, California and 19 other states sued HHS and DHS. The states allege, among other things, that these federal agencies violated (1) the data disclosure limits in the Social Security Act, the Privacy Act, and HIPAA, and (2) the notice-and-comment requirements for rulemaking under the Administrative Procedure Act (APA).

Our amicus brief argues that (1) disclosure of sensitive Medicaid data causes a severe privacy harm to the enrolled individuals, (2) the APA empowers federal courts to block unlawful disclosure of personal data between federal agencies, and (3) the broader public is harmed by these agencies’ lack of transparency about these radical changes in data governance.

A new agency agreement, recently reported by the AP, allows Immigration and Customs Enforcement (ICE) to access the personal data of Medicaid enrollees held by HHS’ Centers for Medicare and Medicaid Services (CMS). The agreement states: “ICE will use the CMS data to allow ICE to receive identity and location information on aliens identified by ICE.”

In the 1970s, in the wake of the Watergate and COINTELPRO scandals, Congress wisely enacted numerous laws to protect our data privacy from government misuse. This includes strict legal limits on disclosure of personal data within an agency, or from one agency to another. EFF sued over DOGE agents grabbing personal data from the U.S. Office of Personnel Management, and filed an amicus brief in a suit challenging ICE grabbing taxpayer data. We’ve also reported on the U.S. Department of Agriculture’s grab of food stamp data and DHS’s potential grab of postal data. And we’ve written about the dangers of consolidating all government information.

We have data protection rules for good reason, and these latest data grabs are exactly why.

You can read our new amicus brief here.

AI image generation — which relies on neural networks to create new images from a variety of inputs, including text prompts — is projected to become a billion-dollar industry by the end of this decade. Even with today’s technology, if you wanted to make a fanciful picture of, say, a friend planting a flag on Mars or heedlessly flying into a black hole, it could take less than a second. However, before they can perform tasks like that, image generators are commonly trained on massive datasets containing millions of images that are often paired with associated text. Training these generative models can be an arduous chore that takes weeks or months, consuming vast computational resources in the process.

But what if it were possible to generate images through AI methods without using a generator at all? That real possibility, along with other intriguing ideas, was described in a research paper presented at the International Conference on Machine Learning (ICML 2025), which was held in Vancouver, British Columbia, earlier this summer. The paper, describing novel techniques for manipulating and generating images, was written by Lukas Lao Beyer, a graduate student researcher in MIT’s Laboratory for Information and Decision Systems (LIDS); Tianhong Li, a postdoc at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL); Xinlei Chen of Facebook AI Research; Sertac Karaman, an MIT professor of aeronautics and astronautics and the director of LIDS; and Kaiming He, an MIT associate professor of electrical engineering and computer science.

This group effort had its origins in a class project for a graduate seminar on deep generative models that Lao Beyer took last fall. In conversations during the semester, it became apparent to both Lao Beyer and He, who taught the seminar, that this research had real potential, which went far beyond the confines of a typical homework assignment. Other collaborators were soon brought into the endeavor.

The starting point for Lao Beyer’s inquiry was a June 2024 paper, written by researchers from the Technical University of Munich and the Chinese company ByteDance, which introduced a new way of representing visual information called a one-dimensional tokenizer. With this device, which is also a kind of neural network, a 256x256-pixel image can be translated into a sequence of just 32 numbers, called tokens. “I wanted to understand how such a high level of compression could be achieved, and what the tokens themselves actually represented,” says Lao Beyer.

The previous generation of tokenizers would typically break up the same image into an array of 16x16 tokens — with each token encapsulating information, in highly condensed form, that corresponds to a specific portion of the original image. The new 1D tokenizers can encode an image more efficiently, using far fewer tokens overall, and these tokens are able to capture information about the entire image, not just a single quadrant. Each of these tokens, moreover, is a 12-digit number consisting of 1s and 0s, allowing for 212 (or about 4,000) possibilities altogether. “It’s like a vocabulary of 4,000 words that makes up an abstract, hidden language spoken by the computer,” He explains. “It’s not like a human language, but we can still try to find out what it means.”

That’s exactly what Lao Beyer had initially set out to explore — work that provided the seed for the ICML 2025 paper. The approach he took was pretty straightforward. If you want to find out what a particular token does, Lao Beyer says, “you can just take it out, swap in some random value, and see if there is a recognizable change in the output.” Replacing one token, he found, changes the image quality, turning a low-resolution image into a high-resolution image or vice versa. Another token affected the blurriness in the background, while another still influenced the brightness. He also found a token that’s related to the “pose,” meaning that, in the image of a robin, for instance, the bird’s head might shift from right to left.

“This was a never-before-seen result, as no one had observed visually identifiable changes from manipulating tokens,” Lao Beyer says. The finding raised the possibility of a new approach to editing images. And the MIT group has shown, in fact, how this process can be streamlined and automated, so that tokens don’t have to be modified by hand, one at a time.

He and his colleagues achieved an even more consequential result involving image generation. A system capable of generating images normally requires a tokenizer, which compresses and encodes visual data, along with a generator that can combine and arrange these compact representations in order to create novel images. The MIT researchers found a way to create images without using a generator at all. Their new approach makes use of a 1D tokenizer and a so-called detokenizer (also known as a decoder), which can reconstruct an image from a string of tokens. However, with guidance provided by an off-the-shelf neural network called CLIP — which cannot generate images on its own, but can measure how well a given image matches a certain text prompt — the team was able to convert an image of a red panda, for example, into a tiger. In addition, they could create images of a tiger, or any other desired form, starting completely from scratch — from a situation in which all the tokens are initially assigned random values (and then iteratively tweaked so that the reconstructed image increasingly matches the desired text prompt).

The group demonstrated that with this same setup — relying on a tokenizer and detokenizer, but no generator — they could also do “inpainting,” which means filling in parts of images that had somehow been blotted out. Avoiding the use of a generator for certain tasks could lead to a significant reduction in computational costs because generators, as mentioned, normally require extensive training.

What might seem odd about this team’s contributions, He explains, “is that we didn’t invent anything new. We didn’t invent a 1D tokenizer, and we didn’t invent the CLIP model, either. But we did discover that new capabilities can arise when you put all these pieces together.”

“This work redefines the role of tokenizers,” comments Saining Xie, a computer scientist at New York University. “It shows that image tokenizers — tools usually used just to compress images — can actually do a lot more. The fact that a simple (but highly compressed) 1D tokenizer can handle tasks like inpainting or text-guided editing, without needing to train a full-blown generative model, is pretty surprising.”

Zhuang Liu of Princeton University agrees, saying that the work of the MIT group “shows that we can generate and manipulate the images in a way that is much easier than we previously thought. Basically, it demonstrates that image generation can be a byproduct of a very effective image compressor, potentially reducing the cost of generating images several-fold.”

There could be many applications outside the field of computer vision, Karaman suggests. “For instance, we could consider tokenizing the actions of robots or self-driving cars in the same way, which may rapidly broaden the impact of this work.”

Lao Beyer is thinking along similar lines, noting that the extreme amount of compression afforded by 1D tokenizers allows you to do “some amazing things,” which could be applied to other fields. For example, in the area of self-driving cars, which is one of his research interests, the tokens could represent, instead of images, the different routes that a vehicle might take.

Xie is also intrigued by the applications that may come from these innovative ideas. “There are some really cool use cases this could unlock,” he says. 

In 2001, MIT became the first higher education institution to provide educational resources for free to anyone in the world. Fast forward 24 years: The Institute has now launched a dynamic AI-enabled website for its non-degree learning opportunities, making it easier for learners around the world to discover the courses and resources available on MIT’s various learning platforms.

MIT Learn enables learners to access more than 12,700 educational resources — including introductory and advanced courses, courseware, videos, podcasts, and more — from departments across the Institute. MIT Learn is designed to seamlessly connect the existing Institute’s learning platforms in one place.

“With MIT Learn, we’re opening access to MIT’s digital learning opportunities for millions around the world,” says Dimitris Bertsimas, vice provost for open learning. “MIT Learn elevates learning with personalized recommendations powered by AI, guiding each learner toward deeper understanding. It is a stepping stone toward a broader vision of making these opportunities even more accessible to global learners through one unified learning platform.”

The goal for MIT Learn is twofold: to allow learners to find what they want to fulfill their curiosity, and to enable learners to develop a long-term relationship with MIT as a source of educational experiences. 

“By fostering long-term connections between learners and MIT, we not only provide a pathway to continued learning, but also advance MIT’s mission to disseminate knowledge globally,” says Ferdi Alimadhi, chief technology officer for MIT Open Learning and the lead of the MIT Learn project. “With this initial launch of MIT Learn, we’re introducing AI-powered features that leverage emerging technologies to help learners discover the right content, engage with it more deeply, and stay supported as they shape their own educational journeys.” 

With its sophisticated search, browse, and discovery capability, MIT Learn allows learners to explore topics without having to understand MIT’s organizational structure or know the names of departments and programs. An AI-powered recommendation feature called “Ask Tim” complements the site’s traditional search and browsing tools, helping learners quickly find courses and resources aligned with their personal and professional goals. Learners can also prompt “Ask Tim” for a summary of a course’s structure, topics, and expectations, leading to more-informed decisions before enrolling.

In select offerings, such as Molecular Biology: DNA Replication and Repair, Genetics: The Fundamentals, and Cell Biology: Transport and Signaling, learners can interact with an AI assistant by asking questions about a lecture, requesting flashcards of key concepts, and obtaining instant summaries. These select offerings also feature an AI tutor to support learners as they work through problem sets, guiding them toward the next step without giving away the answers. These features, Alimadhi says, are being introduced in a limited set of courses and modules to allow the MIT Open Learning team to gather insights and improve the learning experience before expanding more broadly.

“MIT Learn is a whole new front door to the Institute,” says Christopher Capozzola, senior associate dean for open learning, who worked with faculty across the Institute on the project. “Just as the Kendall Square renovations transformed the way that people interact with our physical campus, MIT Learn transforms how people engage with what we offer digitally.”

Learners who choose to create an account on MIT Learn receive personalized course recommendations and can create and curate lists of educational resources, follow their specific areas of interest, and receive notifications when new MIT content is available. They can also personalize their learning experience based on their specific interests and choose the format that is best suited to them.

"From anywhere and for anyone, MIT Learn makes lifelong learning more accessible and personalized, building on the Institute’s decades of global leadership in open learning,” says MIT Provost Anantha Chandrakasan.

MIT Learn was designed to account for a learner’s evolving needs throughout their learning journey. It highlights supplemental study materials for middle schoolers, high schoolers, and college students, upskilling opportunities for early-career professionals, reskilling programs for those considering a career shift, and resources for educators.

“MIT has an amazing collection of learning opportunities, covering a wide range of formats,” says Eric Grimson, chancellor for academic advancement, who oversaw the initial development of MIT Learn during his time as interim vice president for open learning. “The sheer size of that collection can be daunting, so creating a platform that brings all of those offerings together, in an easily searchable framework, greatly enhances our ability to serve learners.”

According to Peter Hirst, senior associate dean for executive education at MIT Sloan School of Management, one of the Institute's incredible strengths is its sheer volume and diversity of expertise, research, and learning opportunities. But it can be challenging to discover and follow all those opportunities — even for people who are immersed in the on-campus experience. MIT Learn, he says, is a solution to this problem.

“MIT Learn gathers all the knowledge and learning resources offered across all of MIT into a learner-friendly, curatable repository that enables anyone and everyone, whatever their interests or learning needs, to explore and engage in the wide range of learning resources and public certificate programs that MIT has to offer and that can help them achieve their goals,” Hirst says.

MIT Learn was spearheaded by MIT Open Learning, which aims to transform teaching and learning on and off the Institute’s campus. MIT Learn was developed with the direction of former provost Cynthia Barnhart, and in cooperation with Sloan Executive Education and Professional Education. During the design phase, OpenCourseWare Faculty Advisory Committee Chair Michael Short and MITx Faculty Advisory Committee Chair Caspar Hare contributed key insights, along with other numerous faculty involved with Open Learning’s product offerings, including OpenCourseWare, MITx, and MicroMasters programs. MIT Learn is also informed by the insights of the Ad Hoc Committee on MITx and MITx Online.

“For over 20 years, MIT staff and faculty have been creating a wealth of online resources, from lecture videos to practice problems, and from single online courses to entire credential-earning programs,” says Sara Fisher Ellison, a member of the Ad Hoc Committee on MITx and MITx Online and the faculty lead for the online MITx MicroMasters Program in Data, Economics, and Design of Policy. “Making these resources findable, searchable, and broadly available is a natural extension of MIT’s core educational mission. MIT Learn is a big, important step in that direction. We are excited for the world to see what we have to offer.”

Looking ahead, MIT Learn will also feature selected content from the MIT Press. As MIT Learn continues to grow, Open Learning is exploring collaborations with departments across the Institute with the goal of offering the fullest possible range of educational materials from MIT to learners around the world.

“MIT Learn is the latest step in a long tradition of the Institute providing innovative ways for learners to access knowledge,” Barnhart says. “This AI-enabled platform delivers on the Institute’s commitment to help people launch into learning journeys that can unlock life-changing opportunities.”

Staying safe whilst dating online should not be the responsibility of users—dating apps should be prioritizing our privacy by default, and laws should require companies to prioritize user privacy over their profit. But dating apps are taking shortcuts in safeguarding the privacy and security of users in favour of developing and deploying AI tools on their platforms, sometimes by using your most personal information to train their AI tools. 

Grindr has big plans for its gay wingman bot, Bumble launched AI Icebreakers, Tinder introduced AI tools to choose profile pictures for users, OKCupid teamed up with AI photo editing platform Photoroom to erase your ex from profile photos, and Hinge recently launched an AI tool to help users write prompts.

The list goes on, and the privacy harms are significant. Dating apps have built platforms that encourage people to be exceptionally open with sensitive and potentially dangerous personal information. But at the same time, the companies behind the platforms collect vast amounts of intimate details about their customers—everything from sexual preferences to precise location—who are often just searching for compatibility and connection. This data falling into the wrong hands can—and has—come with unacceptable consequences, especially for members of the LGBTQ+ community. 

This is why corporations should provide opt-in consent for AI training data obtained through channels like private messages, and employ minimization practices for all other data. Dating app users deserve the right to privacy, and should have a reasonable expectation that the contents of conversations—from text messages to private pictures—are not going to be shared or used for any purpose that opt-in consent has not been provided for. This includes the use of personal data for building AI tools, such as chatbots and picture selection tools. 

AI Icebreakers

Back in December 2023, Bumble introduced AI Icebreakers to the ‘Bumble for Friends’ section of the app to help users start conversations by providing them with AI-generated messages. Powered by OpenAI’s ChatGPT, the feature was deployed in the app without ever asking for their consent. Instead, the company presented users with a pop-up upon entering the app which repeatedly nudged people to click ‘Okay’ or face the same pop-up every time the app is reopened until individuals finally relent and tap ‘Okay.’

Obtaining user data without explicit opt-in consent is bad enough. But Bumble has taken this even further by sharing personal user data from its platform with OpenAI to feed into the company’s AI systems. By doing this, Bumble has forced its AI feature on millions of users in Europe—without their consent but with their personal data.

In response, European nonprofit noyb recently filed a complaint with the Austrian data protection authority on Bumble’s violation of its transparency obligations under Article 5(1)(a) GDPR. In its report, noyb flagged concerns around Bumble’s data sharing with OpenAI, which allowed the company to generate an opening message based on information users shared on the app. 

In its complaint, noyb specifically alleges that Bumble: 

• Failed to provide information about the processing of personal data for its AI Icebreaker feature 
• Confused users with a “fake” consent banner
• Lacks a legal basis under Article 6(1) GDPR as it never sought user consent and cannot legally claim to base its processing on legitimate interest 
• Can only process sensitive data—such as data involving sexual orientation—with explicit consent per Article 9 GDPR
• Failed to adequately respond to the complainant’s access request, regulated through Article 15 GDPR.

AI Chatbots for Dating

Grindr recently launched its AI wingman. The feature operates like a chatbot and currently keeps track of favorite matches and suggests date locations. In the coming years, Grindr plans for the chatbot to send messages to other AI agents on behalf of users, and make restaurant reservations—all without human intervention. This might sound great: online dating without the time investment? A win for some! But privacy concerns remain. 

The chatbot is being built in collaboration with a third party company called Ex-human, which raises concerns about data sharing. Grindr has communicated that its users’ personal data will remain on its own infrastructure, which Ex-Human does not have access to, and that users will be “notified” when AI tools are available on the app. The company also said that it will ask users for permission to use their chat history for AI training. But AI data poses privacy risks that do not seem fully accounted for, particularly in places where it’s not safe to be outwardly gay. 

In building this ‘gay chatbot,’ Grindr’s CEO said one of its biggest limitations was preserving user privacy. It’s good that they are cognizant of these harms, particularly because the company has a terrible track record of protecting user privacy, and the company was also recently sued for allegedly revealing the HIV status of users. Further, direct messages on Grindr are stored on the company’s servers, where you have to trust they will be secured, respected, and not used to train AI models without your consent. Given Grindr’s poor record of not respecting user consent and autonomy on the platform, users need additional protections and guardrails for their personal data and privacy than currently being provided—especially for AI tools that are being built by third parties. 

AI Picture Selection  

In the past year, Tinder and Bumble have both introduced AI tools to help users choose better pictures for their profiles. Tinder’s AI-powered feature, Photo Selector, requires users to upload a selfie, after which its facial recognition technology can identify the person in their camera roll images. The Photo Selector then chooses a “curated selection of photos” direct from users’ devices based on Tinder’s “learnings” about good profile images. Users are not informed about the parameters behind choosing photos, nor is there a separate privacy policy introduced to guardrail privacy issues relating to the potential collection of biometric data, and collection, storage, and sale of camera roll images. 

The Way Forward: Opt-In Consent for AI Tools and Consumer Privacy Legislation 

Putting users in control of their own data is fundamental to protecting individual and collective privacy. We all deserve the right to control how our data is used and by whom. And when it comes to data like profile photos and private messages, all companies should require opt-in consent before processing those messages for AI. Finding love should not involve such a privacy impinging tradeoff.

At EFF, we’ve also long advocated for the introduction of comprehensive consumer privacy legislation to limit the collection of our personal data at its source and prevent retained data being sold or given away, breached by hackers, disclosed to law enforcement, or used to manipulate a user’s choices through online behavioral advertising. This would help protect users on dating apps as reducing the amount of data collected prevents the subsequent use in ways like building AI tools and training AI models. 

The privacy options at our disposal may seem inadequate to meet the difficult moments ahead of us, especially for vulnerable communities, but these steps are essential to protecting users on dating apps. We urge companies to put people over profit and protect privacy on their platforms.

Simply using extra electricity to power some Christmas lights or a big fish tank shouldn’t bring the police to your door. In fact, in California, the law explicitly protects the privacy of power customers, prohibiting public utilities from disclosing precise “smart” meter data in most cases. 

Despite this, Sacramento’s power company and law enforcement agencies have been running an illegal mass surveillance scheme for years, using our power meters as home-mounted spies. The Electronic Frontier Foundation (EFF) is seeking to end Sacramento’s dragnet surveillance of energy customers and have asked for a court order to stop this practice for good.

For a decade, the Sacramento Municipal Utilities District (SMUD) has been searching through all of its customers’ energy data, and passed on more than 33,000 tips about supposedly “high” usage households to police. Ostensibly looking for homes that were growing illegal amounts of cannabis, SMUD analysts have admitted that such “high” power usage could come from houses using air conditioning or heat pumps or just being large. And the threshold of so-called “suspicion” has steadily dropped, from 7,000 kWh per month in 2014 to just 2,800 kWh a month in 2023. One SMUD analyst admitted that they themselves “used 3500 [kWh] last month.”

This scheme has targeted Asian customers. SMUD analysts deemed one home suspicious because it was “4k [kWh], Asian,” and another suspicious because “multiple Asians have reported there.” Sacramento police sent accusatory letters in English and Chinese, but no other language, to residents who used above-average amounts of electricity.

In 2022, EFF and the law firm Vallejo, Antolin, Agarwal, Kanter LLP sued SMUD and the City of Sacramento, representing the Asian American Liberation Network and two Sacramento County residents. One is an immigrant from Vietnam. Sheriff’s deputies showed up unannounced at his home, falsely accused him of growing cannabis based on an erroneous SMUD tip, demanded entry for a search, and threatened him with arrest when he refused. He has never grown cannabis; rather, he consumes more than average electricity due to a spinal injury.

Last week, we filed our main brief explaining how this surveillance program violates the law and why it must be stopped. California’s state constitution bars unreasonable searches. This type of dragnet surveillance — suspicionless searches of entire zip codes worth of customer energy data — is inherently unreasonable. Additionally, a state statute generally prohibits public utilities from sharing such data. As we write in our brief, the Sacramento’s mass surveillance scheme does not qualify for one of the narrow exceptions to this rule. 

Mass surveillance violates the privacy of many individuals, as police without individualized suspicion seek (possibly non-existent) evidence of some kind of offense by some unknown person. As we’ve seen time and time again, innocent people inevitably get caught in the dragnet. For decades, EFF has been exposing and fighting these kinds of dangerous schemes. We remain committed to protecting digital privacy, whether it’s being threatened by national governments – or your local power company.

Related Cases: Asian American Liberation Network v. SMUD, et al.

Let’s say you’re reading a story, or playing a game of chess. You may not have noticed, but each step of the way, your mind kept track of how the situation (or “state of the world”) was changing. You can imagine this as a sort of sequence of events list, which we use to update our prediction of what will happen next.

Language models like ChatGPT also track changes inside their own “mind” when finishing off a block of code or anticipating what you’ll write next. They typically make educated guesses using transformers — internal architectures that help the models understand sequential data — but the systems are sometimes incorrect because of flawed thinking patterns. Identifying and tweaking these underlying mechanisms helps language models become more reliable prognosticators, especially with more dynamic tasks like forecasting weather and financial markets.

But do these AI systems process developing situations like we do? A new paper from researchers in MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and Department of Electrical Engineering and Computer Science shows that the models instead use clever mathematical shortcuts between each progressive step in a sequence, eventually making reasonable predictions. The team made this observation by going under the hood of language models, evaluating how closely they could keep track of objects that change position rapidly. Their findings show that engineers can control when language models use particular workarounds as a way to improve the systems’ predictive capabilities.

Shell games

The researchers analyzed the inner workings of these models using a clever experiment reminiscent of a classic concentration game. Ever had to guess the final location of an object after it’s placed under a cup and shuffled with identical containers? The team used a similar test, where the model guessed the final arrangement of particular digits (also called a permutation). The models were given a starting sequence, such as “42135,” and instructions about when and where to move each digit, like moving the “4” to the third position and onward, without knowing the final result.

In these experiments, transformer-based models gradually learned to predict the correct final arrangements. Instead of shuffling the digits based on the instructions they were given, though, the systems aggregated information between successive states (or individual steps within the sequence) and calculated the final permutation.

One go-to pattern the team observed, called the “Associative Algorithm,” essentially organizes nearby steps into groups and then calculates a final guess. You can think of this process as being structured like a tree, where the initial numerical arrangement is the “root.” As you move up the tree, adjacent steps are grouped into different branches and multiplied together. At the top of the tree is the final combination of numbers, computed by multiplying each resulting sequence on the branches together.

The other way language models guessed the final permutation was through a crafty mechanism called the “Parity-Associative Algorithm,” which essentially whittles down options before grouping them. It determines whether the final arrangement is the result of an even or odd number of rearrangements of individual digits. Then, the mechanism groups adjacent sequences from different steps before multiplying them, just like the Associative Algorithm.

“These behaviors tell us that transformers perform simulation by associative scan. Instead of following state changes step-by-step, the models organize them into hierarchies,” says MIT PhD student and CSAIL affiliate Belinda Li SM ’23, a lead author on the paper. “How do we encourage transformers to learn better state tracking? Instead of imposing that these systems form inferences about data in a human-like, sequential way, perhaps we should cater to the approaches they naturally use when tracking state changes.”

“One avenue of research has been to expand test-time computing along the depth dimension, rather than the token dimension — by increasing the number of transformer layers rather than the number of chain-of-thought tokens during test-time reasoning,” adds Li. “Our work suggests that this approach would allow transformers to build deeper reasoning trees.”

Through the looking glass

Li and her co-authors observed how the Associative and Parity-Associative algorithms worked using tools that allowed them to peer inside the “mind” of language models. 

They first used a method called “probing,” which shows what information flows through an AI system. Imagine you could look into a model’s brain to see its thoughts at a specific moment — in a similar way, the technique maps out the system’s mid-experiment predictions about the final arrangement of digits.

A tool called “activation patching” was then used to show where the language model processes changes to a situation. It involves meddling with some of the system’s “ideas,” injecting incorrect information into certain parts of the network while keeping other parts constant, and seeing how the system will adjust its predictions.

These tools revealed when the algorithms would make errors and when the systems “figured out” how to correctly guess the final permutations. They observed that the Associative Algorithm learned faster than the Parity-Associative Algorithm, while also performing better on longer sequences. Li attributes the latter’s difficulties with more elaborate instructions to an over-reliance on heuristics (or rules that allow us to compute a reasonable solution fast) to predict permutations.

“We’ve found that when language models use a heuristic early on in training, they’ll start to build these tricks into their mechanisms,” says Li. “However, those models tend to generalize worse than ones that don’t rely on heuristics. We found that certain pre-training objectives can deter or encourage these patterns, so in the future, we may look to design techniques that discourage models from picking up bad habits.”

The researchers note that their experiments were done on small-scale language models fine-tuned on synthetic data, but found the model size had little effect on the results. This suggests that fine-tuning larger language models, like GPT 4.1, would likely yield similar results. The team plans to examine their hypotheses more closely by testing language models of different sizes that haven’t been fine-tuned, evaluating their performance on dynamic real-world tasks such as tracking code and following how stories evolve.

Harvard University postdoc Keyon Vafa, who was not involved in the paper, says that the researchers’ findings could create opportunities to advance language models. “Many uses of large language models rely on tracking state: anything from providing recipes to writing code to keeping track of details in a conversation,” he says. “This paper makes significant progress in understanding how language models perform these tasks. This progress provides us with interesting insights into what language models are doing and offers promising new strategies for improving them.”

Li wrote the paper with MIT undergraduate student Zifan “Carl” Guo and senior author Jacob Andreas, who is an MIT associate professor of electrical engineering and computer science and CSAIL principal investigator. Their research was supported, in part, by Open Philanthropy, the MIT Quest for Intelligence, the National Science Foundation, the Clare Boothe Luce Program for Women in STEM, and a Sloan Research Fellowship.

The researchers presented their research at the International Conference on Machine Learning (ICML) this week.

ProPublica is reporting:

Microsoft is using engineers in China to help maintain the Defense Department’s computer systems—with minimal supervision by U.S. personnel—leaving some of the nation’s most sensitive data vulnerable to hacking from its leading cyber adversary, a ProPublica investigation has found.

The arrangement, which was critical to Microsoft winning the federal government’s cloud computing business a decade ago, relies on U.S. citizens with security clearances to oversee the work and serve as a barrier against espionage and sabotage...

Government spacecraft would be available for purchase under a plan being discussed by the Trump administration.

The office created under President Joe Biden to run the $27 billion Greenhouse Gas Reduction Fund has been turned into an oversight arm.

Top Republicans say they're focused on the nuts and bolts of disaster policy in the aftermath of the deadly flash floods.

The International Court of Justice heard testimony from more than 100 nations and organizations in the lead-up to its decision.

The National Association of Counties passed a resolution opposing efforts to limit climate lawsuits against the oil and gas industry.

Democrats pared back one of the state's preeminent policies to restore trust with voters frustrated by the high cost of living.

Parliament’s centrist and left-wing forces have vowed to try to stop Ondřej Knotek from stalling work on the 2040 target.

Asian countries are offering to buy more U.S. liquefied natural gas as a way to alleviate tensions over American trade deficits and forestall higher tariffs.

Interest in financial products claiming to target environmental, social and governance goals is flagging amid politically motivated attacks.

Doing your taxes can feel like a very complicated task. Even so, it might be less intricate than trying to make sense of what people think about taxes.

Several years ago, MIT political scientist Andrea Campbell undertook an expansive research project to understand public opinion about taxation. Her efforts have now reached fruition, in a new book uncovering many complexities about attitudes toward taxes. Those complexities include a central tension: In the U.S., most people say they support the principle of progressive taxation — in which higher earners pay higher shares of their income. Yet people also say they prefer specific forms of taxes that are regressive, hitting lower- and middle-income earners relatively harder.

For instance, state sales taxes are considered regressive, since people who make less money spend a larger percentage of their incomes, meaning sales taxes eat up a larger proportion of their earnings. But a substantial portion of the public still finds them to be fair, partly because the wealthy cannot wriggle out of them.

“At an abstract or conceptual level, people say they like progressive tax systems more than flat or regressive tax systems,” Campbell says. “But when you look at public attitudes toward specific taxes, people’s views flip upside down. People say federal and state income taxes are unfair, but they say sales taxes, which are very regressive, are fair. Their attitudes on individual taxes are the opposite of what their overall commitments are.”

Now Campbell analyzes these issues in detail in her book, “Taxation and Resentment,” just published by Princeton University Press. Campbell is the Arthur and Ruth Sloan Professor of Political Science at MIT and a former head of MIT’s  Department of Political Science.

Filling out the record

Campbell originally planned “Taxation and Resentment” as a strictly historically-oriented look at the subject. But the absence of any one book compiling public-opinion data in this area was striking. So, she assembled data going back to the end of World War II, and even designed and ran a couple of her own public research surveys, which help undergird the book’s numbers.

“Political scientists write a lot about public attitudes toward spending in the United States, but not so much about attitudes toward taxes,” Campbell says. “The public-opinion record is very thin.”

The complexities of U.S. public opinion on taxes are plainly linked to the presence of numerous forms of taxes, including federal and state income taxes, sales taxes, payroll taxes, estate taxes, and capital gains taxes. The best-known, of course, is the federal income tax, whose quirks and loopholes seem to irk citizens.

“That really seizes people’s imaginations,” Campbell says. “Keeping the focus on federal income tax has been a clever strategy among those who want to cut it. People think it’s unfair because they look at all the tax breaks the rich get and think, ‘I don’t have access to those.’ Those breaks increase complexity, undermine people’s knowledge, heighten their anger, and of course are in there because they help rich people pay less. So, there ends up being a cycle.”

That same sense of unfairness does not translate to all other forms of taxation, however. Large majorities of people have supported lowering the estate tax, for example, even though the threshold at which the federal estate tax kicks in — $13.5 million — applies to very few families.

Then too, the public seems to perceive sales taxes as being fair because of the simplicity and lack of loopholes — an understandable view, but one that ignores the way that state sales taxes, as opposed to state income taxes, place a bigger burden on middle-class and lower-income workers.

“A regressive tax like a sales tax is more difficult to comprehend,” Campbell says. “We all pay the same rate, so it seems like a flat tax, but as your income goes up, the bite of that tax goes down. And that’s just very difficult for people to understand.”

Overall, as Campbell details, income levels do not have huge predictive value when it comes to tax attitudes. Party affiliation also has less impact than many people might suspect — Democrats and Republicans differ on taxes, though not as much, in some ways, as political independents, who often have the most anti-tax views of all.

Meanwhile, Campbell finds, white Americans with heightened concerns about redistribution of public goods among varying demographic groups are more opposed to taxes than those who do not share those redistribution concerns. And Black and Hispanic Americans, who may wind up on the short end of regressive policies, also express significantly anti-tax perspectives, albeit while expressing more support for the state functions funded by taxation.

“There are so many factors and components of public opinion around taxes,” Campbell says. “Many political and demographic groups have their own reasons for disliking the status quo.”

How much does public opinion matter?

The research in “Taxation and Resentment” will be of high value to many kinds of scholars. However, as Campbell notes, political scientists do not have consensus about how much public opinion influences policy. Some experts contend that donors and lobbyists essentially determine policy while the larger public is ignored. But Campbell does not agree that public sentiment amounts to nothing. Consider, she says, the vigorous and successful public campaign to lower the estate tax in the first decade of the 2000s.

“If public opinion doesn’t matter, then why were there these PR campaigns to try to convince people the estate tax was bad for small businesses, farmers, and other groups?” Campbell asks. “Clearly it’s because public opinion does matter. It’s far easier to get these policies implemented if the public is on your side than if the public is in opposition. Public opinion is not the only factor in policymaking, but it’s a contributing factor.”

To be sure, even in the formation of public opinion, there are complexities and nuance, as Campbell notes in the book. A system of progressive taxation means the people taxed at the highest rate are the most motivated to oppose the system — and may heavily influence public opinion, in a top-down manner.

Scholars in the field have praised “Taxation and Resentment.” Martin Gilens, chair of the Department of Public Policy at the University of California at Los Angeles, has called it an “important and very welcome addition to the literature on public attitudes about public policies … with rich and often unexpected findings.” Vanessa Williamson, a senior fellow at the Brookings Institution, has said the book is “essential reading for anyone who wants to understand what Americans actually think about taxes. The scope of the data Campbell brings to bear on this question is unparalleled, and the depth of her analysis of public opinion across time and demography is a monumental achievement.”

For her part, Campbell says she hopes people in a variety of groups will read the book — including policymakers, scholars in multiple fields, and students. Certainly, she thinks, after studying the issue, more people could stand to know more about taxes.

“The tax system is complex,” Campbell says, “and people don’t always understand their own stakes. There is often a fog surrounding taxes.”

Beautiful photo.

Difficult to capture, this mysterious, squid-shaped interstellar cloud spans nearly three full moons in planet Earth’s sky. Discovered in 2011 by French astro-imager Nicolas Outters, the Squid Nebula’s bipolar shape is distinguished here by the telltale blue emission from doubly ionized oxygen atoms. Though apparently surrounded by the reddish hydrogen emission region Sh2-129, the true distance and nature of the Squid Nebula have been difficult to determine. Still, one investigation suggests Ou4 really does lie within Sh2-129 some 2,300 light-years away. Consistent with that scenario, the cosmic squid would represent a spectacular outflow of material driven by a ...

Amid a wave of lawsuits targeting how AI companies use copyrighted works to train large language models that generate new works, a peculiar provision of copyright law is suddenly in the spotlight: Section 1202 of the Digital Millennium Copyright Act (DMCA). Section 1202 restricts intentionally removing or changing copyright management information (CMI), such as a signature on a painting or attached to a photograph. Passed in 1998, the rule was supposed to help rightsholders identify potentially infringing uses of their works and encourage licensing.

Open AI and Microsoft used code from Github as part of the training data for their LLMs, along with billions of other works. A group of anonymous Github contributors sued, arguing that those LLMs generated new snippets of code that were substantially similar to theirs—but with the CMI stripped. Notably, they did not claim that the new code was copyright infringement—they are relying solely on Section 1202 of the DMCA. Their problem? The generated code is different from their original work, and courts across the US have adopted an “identicality rule,” on the theory that Section 1202 is supposed to apply only when CMI is removed from existing works, not when it’s simply missing from a new one.

It may sound like an obscure legal question, but the outcome of this battle—currently before the Ninth Circuit Court of Appeals—could have far-reaching implications beyond generative AI technologies. If the rightholders were correct, Section 1202 effectively creates a freestanding right of attribution, creating potential liability even for non-infringing uses, such as fair use, if those new uses simply omit the CMI. While many fair users might ultimately escape liability under other limitations built into Section 1202, the looming threat of litigation, backed by risk of high and unpredictable statutory penalties, will be enough to pressure many defendants to settle. Indeed, an entire legal industry of “copyright trolls” has emerged to exploit this dynamic, with no corollary benefit to creativity or innovation.

Fortunately, as we explain in a brief filed today, the text of Section 1202 doesn’t support such an expansive interpretation. The provision repeatedly refers to “works” and “copies of works”—not “substantially similar” excerpts or new adaptations—and its focus on “removal or alteration” clearly contemplates actions taken with respect to existing works, not new ones. Congress could have chosen otherwise and written the law differently. Wisely it did not, thereby ensuring that rightsholders couldn’t leverage the omission of CMI to punish or unfairly threaten otherwise lawful re-uses of a work.

Given the proliferation of copyrighted works in virtually every facet of daily life, the last thing any court should do is give rightsholders a new, freestanding weapon against fair uses. As the Supreme Court once observed, copyright is a “tax on readers for the purpose of giving a bounty to writers.” That tax—including the expense of litigation—can be an important way to encourage new creativity, but it should not be levied unless the Copyright Act clearly requires it.