I have a blog post, here on this blog called "Find song by lyrics". It's very popular and has, over the years, 28k approved comments posted.
Now, I've built an AI tool that attempts to edit these comments by correcting spelling and punctuation, without changing the meaning. It looks like this:
When I'm moderating, every. single. comment, this is more or less what I myself do, if I can be bothered. I don't want to take away the message, but I think it looks and reads a lot of better if I correct the spelling and punctuations. It's super common that people write sentences like this:
im looking for a song.a lady with a lowe voice that sound happy,and i cant find anywhere
I correct that to:
I'm looking for a song. A lady with a low voice that sound happy, and I can't find anywhere.
It's just spelling and punctuation. The sentence structure isn't changed. It's still written by a human, but we can assume a human that didn't have time to correct the little details. Also, a lot of blog post comments are written by people where English is not their native language so it's less expected to be perfect.
The implementation
The way I implemented this was using the OpenAI API, in Python, using litellm, and because they take quite long to compute, they run in an asynchronous message queue. The prompts I wrote to implement it are as follows:
- "You are a helpful editor that reads blog post comments and corrects grammar and punctuations."
- "You have to look for common spelling mistakes, lack of spaces after full stops, incorrect capitalization."
- "Your job is to rewrite the comment without changing the meaning, but correcting any grammar and punctuation mistakes. Only return the rewritten comment and nothing else. Avoid using Unicode quotation marks, use regular ASCII quotes instead."
- "Here is the comment:\n\n```{comment_escaped}```"
I'm not an expert at prompt engineering and first to admit that I have a lot to learn. Because I don't really trust it yet fully, I'm manually skimming all comments first before I allow AI to attempt the rewrite.
Comments
Perhaps you can correct your own punctuation for these editorialised [sic¹] comments.
Any type of content change associated with somebody without making it obvious it has been fiddled with is just bizarre to me.
Here's a helpful reference:
https://www.thepunctuationguide.com/brackets.html
¹: I am British and write in British English. I hope you will not helpfully correct my mistake!
For elegant and eloquent comments, I would never. It's more for the egregiously poorly written comments.
One optimization you could make is to send the sentences as they come. You could watch for periods, when a sentence is complete, send it to the llm and get the response. Then when you hit the edit button it has results you can view almost immediately and the ones that havent finished yet can stream in
I bet that's how Grammarly does things. Lots of clever caching of tokenized sentences because it's "just" a (very fancy) application on top of an LLM.
(In my case, I don't actually correct blog post comments until long after they've been created)