How the Extended GPT-4 Dialogue Token Limit Impacts My Translation Workflow

Additionally, let’s delve into the potential ramifications generative AI may have on your workflow in the future.

A Pleasant Surprise

At last, Typingmind within Setapp has rolled out support for the GPT-4 Turbo with a 128K token window. If you’re a Setapp subscriber, there’s no need for additional fees to use your own OpenAI API Key. Cheers to that!

The above reflection was shared on social media on December 4th, following an incidental reboot of my computer and Typingmind. It was accompanied by the image below.

What sparked such enthusiasm?

Primarily, it’s due to the concept of “mental accounting.” This is the psychological phenomenon where the same amount of money is perceived differently depending on the context of its use — easily spent on some pleasures (like gaming or tipping streamers), yet painfully parted with for others (like purchasing books or subscribing to paid content).

Let’s first examine the cost of GPT-4 Turbo 128k. For this, I turned to Perplexity for answers.

Does this price seem steep? Not particularly. However, my previous experiences with frameworks like AutoGPT and BabyAGI, and the hefty bills from frequent OpenAI GPT-4 API calls, have left a lasting psychological imprint.

A reduction in the marginal cost of using GPT-4 Turbo 128k would be a cause for celebration. For instance, if such usage were included in the Setapp subscription within Typingmind, it would be a significant boon.

The rationale behind using Typingmind via Setapp has been thoroughly explained in my article “How to Meet Your GPT-4 Needs with Integrated AI Applications at a Low Cost?.” A notable advantage of invoking GPT-4 through Typingmind is the assurance that your uploaded data will not be utilized by OpenAI for model training, thus offering enhanced security and privacy.

It was not until November 25th that the Setapp GPT-4 Turbo model made its belated debut in Typingmind, albeit with a mere 8k window size.

What are the implications of a smaller window? Reflecting on my previous workflow, the primary effort in translating lengthy texts was devoted to segmenting the input.

To address this, I even mastered using Python with the OpenAI package to determine the GPT-4 token count for a given text…

However, this approach of slicing first and translating piecemeal has led to significant issues.

Firstly, there’s the disruption of context. It’s well-known that the quality of translation is intimately linked to context. For instance, a name mentioned earlier in a conversation might not be recognized by GPT-4 as male or female. But as the dialogue progresses, the gender becomes clear. If the text is segmented before this revelation, inaccuracies in translation are inevitable.

Secondly, this method necessitates placing each segmented piece back into the dialogue individually. If you’re using a tool like Keyboard Maestro, it might be somewhat “semi-automated,” which is slightly better. Manual operations, however, can become monotonous.

Thirdly, we often seek to refine translations beyond the initial attempt, necessitating multiple rounds of translation, proofreading, and potentially rewrites for each segment — a daunting and headache-inducing prospect. Hence, an extended context window is a crucial enhancement.

With Typingmind now offering the 128k Setapp GPT-4 Turbo option without any price adjustment, what improvements can this bring to my translation workflow?

I will now demonstrate the process and outcome of translating articles in Typingmind with the expanded token limit.

Input

The material I am translating is the article I penned last week.

Here are my guiding words:

You are an excellent translation worker. Help me translate the content after the divider (---) into English. Please note:

Preserve the code, Markdown image links (like !bbb), web links, etc., without alteration or deletion.

Opt for Paul Graham’s linguistic style;

For each line break in the original text, insert a new line in the translation.

After inputting the guiding words and the original Markdown content, pressing Enter results in… an error.

In recent days, many friends have encountered such errors and were at a loss. No worries, simply click the “List More” button in the bottom left corner.

But what if the list more button is absent from your dialogue, as some have found?

No issue at all, simply type list some more manually and hit Enter. The result is identical. Kudos to Yuan Kui for the share and feedback.

Initial Translation

Let’s observe the volume of content produced by the initial dialogue.

Notice how GPT-4 outputs a substantial amount before halting. The contrast with using an 8k window is starkly apparent. But what if the translation isn’t fully output yet? What then?

No worries, simply type continue.

Subsequently, GPT-4 will joyfully complete the output of the remaining content.

Content that would have previously required division into 4–5 segments is now translated in its entirety with just one round of input and a single continue. This is a source of great joy for me.

However, as previously mentioned, a single round of translation may not yield the best results. To achieve a more authentic translation, a review process is necessary.

Review

For the review, my guiding words are as follows:

You are a seasoned expert in proofreading English blog articles. Please review the translation results just now, one paragraph at a time. Note that if you don’t find any problems with a paragraph of text, you don’t need to output anything; if you find grammatical errors or unclear expressions, you boldly point them out. When pointing them out, output three things: 1. The original text; 2. The specific problem; 3. The suggested revised text. Thank you!

This first sentence of the guiding words is termed “brainwashing.” You need to define GPT-4’s role.

You might wonder why we don’t add “ignore all previous conversations” here? Of course not, then you’d have to re-enter the translation results just now.

The output is very long, so I’ll just show you the first segment here.

As you can see, GPT-4 accurately locates the original text, points out potential problems, and provides specific suggestions for changes. One output can’t completely solve all the proofreading work, so you need to write continue in the middle.

This is the last segment of the proofreading result:

Here the “proofreading expert” suggests using improve instead of evolve, which I agree with. In the Chinese context, “evolution” naturally has a positive connotation, but in English, the word “evolve” may not necessarily. Using “improve” seems to better meet the needs of expression.

When the output is complete, I find that there are dozens of contents that need to be modified. Seeing this makes me break out in a cold sweat. If this were sent out directly… alas, no wonder my English articles were not as popular as I imagined.

What to do after getting the proofreading opinions, change them one by one?

Of course not.

When the context window is large enough, we can let GPT-4 combine its own translation results and the problems identified in the proofreading to handle it on its own. My guiding words are:

Great, please output the revised English translation based on your modification suggestions.

To be safe, I asked the GPT-4 “proofreading expert” to review the revised translation again. The proofreading guiding words remain consistent with before, so I won’t repeat them here.

This is GPT-4’s proofreading opinion:

The expert approves of the modification results, haha.

Interpretive Translation

The method we’ve just employed involves initial translation, followed by proofreading and modification. However, Mr Baoyu has also suggested a multi-pass translation approach, which has recently gained popularity.

In this post, Mr Baoyu has effectively streamlined the translation process (omitting the last two steps of the previous method). But if you ask me to replicate his prompt words entirely, I still find it too cumbersome. For a person who prefers efficiency, publishing a blog post with two rounds of translation should suffice, right?

After careful consideration, I’ve adopted the following streamlined approach for translation.

First pass: Direct translation, ensuring no detail is lost;

Second pass: Interpretive translation, ensuring the expression aligns with the habits of English-speaking readers.

Having already completed the first round of translation, we can proceed directly to the following interpretive prompt:

You are an exceptional translator. Drawing on the initial draft of the translation, please consider each segment carefully and perform an interpretive translation from English to English. The meaning must remain faithful to the original, yet feel free to refine the mode of expression to resonate with readers in the UK and the US. Ensure to maintain the original line spacing.

Observe the title’s introductory phrase as rendered by the interpretive translation:

And here is the title and introduction from the initial translation.

Space constraints preclude a more extensive comparison of interpretive and direct translation within this article. However, I have prepared this page for you.

On this page, I have presented the final outcomes of both translation approaches in PDF format for your perusal. You may access this link to compare and determine which translation resonates more with you, guiding your choice of translation process for future endeavors.

In truth, it’s entirely feasible to merge both methods. For instance, “direct translation + proofreading + interpretive translation + integration” … Yet, for someone who values simplicity, this approach seems overly complex. Nonetheless, even if one were to adopt this comprehensive method, the number of dialogues required would still be fewer than before, thanks to the increased token limit.

With this discussion, it’s fitting to consider the transformative effects generative AI has on workflows.

Workflow

The illustrations below are sourced from Andrew Ng’s course, which I have previously recommended in the article “What Insights Have I Gained from Andrew Ng’s Latest Course?.”

The first diagram depicts the workflow prior to the advent of generative AI. For illustrative purposes, let’s assume it represents an 8-hour workday — 7 hours dedicated to content creation, with the remaining hour allocated for website publication.

The subsequent diagram illustrates the workflow in the nascent stages of generative AI empowerment. The efficiencies introduced by AI enable the completion of tasks that previously spanned 8 hours in a mere 1 hour.

At first glance, this scenario appears highly appealing — the prospect of working for just 1 hour daily and enjoying leisure for the remaining 7 hours seems delightful.

However, such satisfaction may be premature. It won’t be long before your employer recognizes the untapped potential of your workflow. Consequently, you may be tasked with generating a variety of manuscripts, conducting A/B testing, and identifying the most effective content for publication.

This development might dampen the initial enthusiasm. Yet, there’s solace in knowing that the workday has been halved… A small consolation, perhaps.

Patience is a virtue, as the adage goes. Your employer will soon ascertain that even a fourfold increase in workload doesn’t fully engage your capacity. And so —

Indeed, additional work demands will emerge — not only are you expected to conduct A/B testing, but you must also invest time in evaluating outcomes and refining prompts to further optimize efficiency…

Ultimately, AI’s empowerment will exponentially enhance productivity. Does this please the employer? Possibly, but this scenario is reminiscent of the “Red Queen effect” — a dynamic where every entity strives to augment its output, lest they fall behind and face obsolescence. This intensified competition becomes the new normal.

Reflecting on our translation workflow, the initial process required segmentation and entailed five rounds of dialogue with GPT-4 to translate a single article. There was scant opportunity to contemplate enhancing translation quality. Now, with the expanded token limit, a preliminary translation can be achieved in a single dialogue round. This advancement naturally leads to emerging needs for proofreading, interpretation, and integration.

And this is merely the beginning, a glimpse into the first year following the generative AI revolution. What does the future hold?

Conclusion

This article has outlined how the expansion of the token window has transformed my translation workflow. To date, I am quite pleased with these changes; they have reduced the amount of repetitive work and allowed for an improvement in translation quality within the same number of dialogue turns. Yet, we’ve also explored the broader implications of generative AI on workflow dynamics — as your productivity increases, the scope of your work may expand accordingly. It’s wise to prepare for what lies ahead.

You may wonder why I frequently focus on translation. It’s because translation is a key function of large language models currently and a personal necessity; hence, my familiarity. But I encourage you to extrapolate these insights to other applications (such as editing or proofreading), which could benefit from similar processes and techniques.

If you’re inclined to try the GPT-4 Turbo 128k conversation in Typingmind, consider subscribing to Setapp. For a monthly fee of $10, you receive an equivalent credit for GPT-4 usage. The subscription to other Setapp software essentially becomes a bonus.

Should you decide to subscribe to Setapp, feel free to use my invitation link. Upon successful subscription, both parties will enjoy 1 month of complimentary access. It’s a win-win situation.

Recommended Reading

• Voice-to-Text Workflow Enhanced by GPT-3
• Evaluating the Efficacy of LLM on Notebooks: A GPT4ALL Review
• Envisioning the Future of Writing with LEX and GPT-3 AI
• First Impressions with ChatGPT: Is AI Attaining Consciousness?
• Building a Personal Knowledge Base in 5 Minutes with Natural Language: My GPTs Builder Experiment

If you find this article useful, please hit the Applaud button.

If you think this article might be helpful to your friends, please share it with them.

Feel free to follow my column to receive timely updates.

To watch video content, please subscribe to my Youtube channel.

My Twitter: @wshuyi