A few months ago, I wrote about how I tried ChatGPT on my own writing. At the time, my main conclusion was that large-scale language models (LLMs) were unable to produce writing at the level of a professional writer. The tone was off, the writing style was not my style, the model repeated facts or placed them in the wrong sections, and I ended up rewriting things multiple times.
But I also predicted that the LLM would improve me further, eventually reaching a level where I could become a very efficient co-pilot for a variety of tasks, including writing, so although I didn’t adopt the LLM as my regular writing tool at the time, I began to adjust my writing process in preparation for the future.
It’s been a few months since I wrote this article, and LLM continues to advance at an impressive rate. Fine-tuning LLM has become much easier thanks to low-code/no-code tools and the GPT fine-tuning assistant.
The 1 million token context window makes it much easier to customize LLMs without tweaking them. New generations of models are increasingly better at following instructions, and they are collecting rich personal datasets about how to accomplish different tasks, which can be used to enhance LLMs.
Over the past few months, I’ve experimented extensively with different prompting techniques in GPT-4o, Claude 3/3.5, and Gemini 1.5 Pro, and LLM has now become an integral part of my writing practice, greatly benefiting some of my articles.
There are many more complex ways to use LLMs, including writing draft code, debugging software, and setting up complex software stacks. However, I chose this simple example because it shows that even someone without a computer science or software engineering background can get started using LLMs and generative AI.
The main lesson I learned is that the LLM field is evolving very rapidly. Always try new tools, models, and techniques. Even if the technology isn’t ready to solve your problem, you can see where things are heading and what the technology can achieve in the future. This allows you to start preparing your thinking, workflows, and data to take advantage of current and future models.
That said, here’s how I adapted my writing workflow to take advantage of the LLM: You can apply the same process to any task that has inputs and results.
Examples of using the LLM in writing
I read many papers every week and write reviews of some that could have a significant impact on enterprise and consumer applications. Previously, my process was a time-consuming process of first reading the paper, highlighting the important parts, and then reviewing the highlights and writing an article.
After experimenting with LLM, I started thinking about developing a more structured workflow that could leverage generative AI. I changed my process to reading papers and copying highlights into a Word document. I then categorized my notes into a few key sections: Challenges, Solutions, and Actionable Results.
It became much easier to refer to my notes while I was writing an article. At the same time, I started developing a dataset of my notes and the resulting articles that I could use to customize LLM later.
After trying out the 12 examples, I decided to see if I could get a JD to write a proper draft review based on my notes, after which I would edit the result and add any comments that were not included in the thesis or notes.
I tried several prompt formats with GPT-4, Claude, and Gemini, and the format that worked best was starting with the roll command.
You are an expert writer of articles on AI research papers, writing in a tone that is easy to read and understand for someone who is technically savvy but not necessarily an AI researcher or ML engineer. I would like to provide you with my notes on the paper and ask you to draft it.
Then I added an instruction section with a command like this:
– Write an article using thesis notes
– Make sure the problem addressed, the solution presented, and the experimental results are described in the paper
– Simplify highly technical topics in a way that is understandable to anyone with a basic understanding of AI and machine learning
– Write concise sentences and use the active voice
– Articles should be between 800-1200 words
– I never use “inflect” (use). I hate it. Instead I write “use.”
Then, we started adding examples of in-context learning in the following format:
Note:
[notes from paper]
+++++
article:
[article text in markdown format]
By adding four more examples, we started to see interesting results. Gemini 1.5 Pro in particular proved to be better than other models at mimicking my writing style. With a context window of 2 million tokens, Gemini is very good at handling long prompts with many in-context learning examples. Also, Google AI Studio has a very intuitive user interface for creating prompt templates.
Gemini did a great job of avoiding repeating facts, staying concise, sticking to word range, and simplifying some of the technical terminology contained in my paper notes. The results weren’t perfect, and I spent a fair bit of time editing the results. But what I got was a condensed version of my notes, with added context that was easy to match with my writing style.
A major part of my job is to read papers and pick notes. This is especially important because the model may make mistakes or misinterpret parts of the paper. While editing, I regularly review my notes to ensure the facts are correct.
Improving the LLM Application
I continued to iterate and refine the prompts. For example, in an early version of my prompt, after editing the drafts generated by the model, I added quotes from papers and occasional interviews with the authors to the articles. Additional editing was necessary to ensure that the quotes did not repeat facts already mentioned in the articles. I began inserting the papers and author quotes into my notes with a special prefix and added instructions to the prompt to use the quotes verbatim.
– Lines marked “Quoted from paper” may be directly quoted and attributed to the authors.
– Lines marked “Comment from the author” should be from the author interviewed for the article.
If you enjoyed this article, please support TechTalks with a paid subscription (which gives you access to exclusive posts).
This further improved the process and reduced the time needed to edit articles. As we experimented with different AI tools, we also found ways to improve other aspects of our writing.
For example, one of the most time-consuming parts of the writing process is capturing key quotes from interviews. Currently, I transcribe interviews in OpenAI Whisper, upload the entire script to Perplexity Pro, and ask the model to extract quotes for different parts of the article. I then add those quotes to my notes before running the prompts in Gemini.
We will continue to improve our process and prompts. For example, many papers contain technical terms and concepts that need to be explained to a wider audience. Many of these concepts are related to recent research that are not included in the parametric memory of the LLM. We are considering creating a prompt pipeline that will extract concepts that need explanation from papers. We could then use a search augmentation generation (RAG) system to create draft explanatory statements that can be incorporated into papers.
We will also experiment with fine-tuning LLM on our dataset. Currently, the input prompt is around 20,000 tokens. We would like to see if we can reduce this to just instructions and notes for new articles, which would make the process scalable and cost-effective.
In a previous essay, I discussed how generative AI will impact writing. The gist of that article was that while some tasks will be automated, AI will remain a tool in the hands of creative humans. I still stand by that assertion: AI can be a tool to help us write better. Sometimes, it helps us automate parts of the writing process. Sometimes, like in this article, you’ll want to type every word yourself. Humans are the writers, and AI is the “writing tool.”
Advance
When I talk to different people, I’m surprised at how few people outside of the tech industry are experimenting with AI. My point in this article is to get out there and try some AI. Just working with the tools will build a mental framework of where the industry is heading and you can start thinking about how to automate mundane tasks at work.
Scientists don’t agree on whether and how AI will eliminate the need for human labor, but most agree that AI tools are becoming increasingly useful, and that humans with AI will undoubtedly perform better than those without.
Share this article with your friends who think AI is not relevant to their jobs.
