Guardian Angels: LLM Personalization for Productivity and Se

I propose an approach for highly personalized LLMs, for near-future productivity gains and personal info/cybersecurity against increasingly powerful LLMs: they should, in the spirit of uploading, try to emulate the user’s values and preferences in order to amplify the principal—not replace them. I discuss a package of techniques and proposals to accomplish such ‘guardian angels’; dynamic evaluation of LLMs combined with active learning and elicitation and heavy inner-monologue search/data-augmentation.

Amitav's Commentary

Summary of Gwern's take in my own words: "We should create LLMs that are personalized to each person to the extent that they can act as a stand-in for that person." The rest of the essay is mostly discussing the mechanisms that would enable that and/or the things that would enable. Gwern refers to this hypothetical LLM as a "guardian angel." For the rest of this blog post I will use the word "agent" to mean what Gwern means by "guardian angel."

Here are my major criticisms of this work:

I think that they abstract away many of the difficulties with making guardian angels work as engineering problems, but I believe that many of the issues that they point out with LLMs (inability to differentiate between data and instructions, lack of creativity, poor memory, etc.) are also engineering problems;
More philosophically, I think that their thesis of training an LLM to think like a person is very difficult because it's very difficult to get somebody's thoughts.

I think there are quite a few interesting things in this though:

Their comments on brain imitation learning (i.e. augmenting textual data with fMRI readings) were interesting. Here are some further reading they provide: Schultz et al. 2019, Liu et al. 2023;
Their comments on generalization (C-f "influence functions") seemed instinctively wrong, but the way in which they phrased their understanding leads me to believe that I am likely wrong on that topic. This section seems quite key to their thesis, however.

Overall it was a very nice, well-written, well-researched essay.
A more philosophical objections I have is around the principle of the guardian angel being to "[e]nhance" the principal. I think this sort of system is uniquely well-suited to people late in their career. The way they describe the system is that it would do what Gwern does to Gwern's standards. For me personally, because I am relatively young, I very rarely have the requisite skills to do a project prior to doing the project, nor do I have standards that are calibrated to assess the output of a project that a hypothetical guardian angel would complete. For me, the process of working is the process of learning. My job primarily is to learn quickly, and the output of that learning is a project or a paper or a product or a system. One open question I have is what a system like guardian angels would look like for folks early in their career; more concretely, how does one have agents that are meant to enhance a principal complete a task autonomously, if the principal does not have the necessary skills to evaluate the output of that task.

Guardian Angels: LLM Personalization for Productivity and Security

Abstract

Amitav's Commentary