Free AI web copilot to create summaries, insights and extended knowledge, download it at here

Abstract

st’s advanced concurrency features provide excellent tooling for efficient and safe multi-threaded programming, maximizing your LLM’s throughput potential.</li><li><b>Web Ecosystem:</b> While Rust may be newer relative to languages like Python and JavaScript, its web development ecosystem is growing rapidly. Frameworks like Actix Web and Rocket offer mature solutions for building high-performance REST APIs.</li><li><b>Cross-Platform Compatibility:</b> Applications built with Rust can easily compile to run on virtually any operating system (Windows, Linux, macOS, etc.). This versatility is a tremendous advantage in deployment scenarios.</li></ol><h1 id="0df0">Let’s set the stage</h1><figure id="d2d7"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*AcpA4MkKboaPY0ONHHsN2g.jpeg"><figcaption></figcaption></figure><p id="4fe8">To interact with LLMs from Rust programs, there are a few primary methods:</p><ol><li><b>API Clients:</b> Many LLM services provide readily available REST APIs. Rust offers excellent HTTP client libraries, such as <code>reqwest</code>, to facilitate seamless communication with these APIs.</li><li><b>Model Hosting:</b> If you need low-latency or offline access, consider hosting language models directly within your Rust server. Rust bindings exist for popular frameworks like ONNX Runtime, allowing you to load and execute models locally.</li><li><b>Hybrid Approaches:</b> In some cases, a combination of the above approaches might be optimal. Your Rust server could interact with an external API when dealing with larger, more computationally intensive LLMs, while hosting smaller models locally for real-time tasks.</li></ol><h1 id="39fa">Our approach</h1><figure id="5b02"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*yxk1qTnQ9WfTTwKl-p20lg.jpeg"><figcaption></figcaption></figure><p id="c74d">In this design brainstorming session, we’ll outline the conceptual framework and key components for building a Rust-based REST server aimed at serving Language Model (LM) requests efficiently. Our goal is to design a scalable and performant server architecture that can handle various LM-related functionalities such as chat interactions, health checks, and version information retrieval.</p><h1 id="c75f">Problem Definition</h1><p id="8384"><b>Goal:</b> Establish a clear objective for our server. Possibilities include:</p><ul><li>Providing a central point of access and control for one or more large language models.</li><li>Offering an API layer for other applications to leverage LLM capabilities easily.</li><li>Abstracting away platform-specific LLM details behind a simple REST interface.</li></ul><h1 id="93e8">Target Users:</h1><p id="822d">Who are we building this server for?</p><ul><li>Developers building LLM-powered applications.</li><li>Data scientists conducting experiments with LLMs.</li><li>Int

Options

ernal services within an organization that need LLM functionality.</li></ul><h1 id="40ab">Design Thinking for a Rust LLM REST Server</h1><ol><li>Project Structure:</li></ol><p id="692a">We’ll start by defining the overall project structure, including modules, dependencies, and project organization. This involves setting up a Cargo-based project with appropriate dependencies for handling HTTP requests, JSON serialization, and any required LM-related functionality.</p><p id="08d8">2. Endpoint Design:</p><p id="347b">Next, we’ll design the REST API endpoints that our server will expose. Key endpoints may include:</p><ul><li><code>/api/query</code>: Endpoint for handling chat interactions with the Language Model.</li><li><code>/api/health</code>: Endpoint for performing health checks to ensure the server is running smoothly.</li><li><code>/api/app/version</code>: Endpoint for retrieving version information of the server application.</li></ul><p id="7e21">Each endpoint will have specific request/response formats and logic for handling incoming requests and generating appropriate responses.</p><p id="5660">3. Language Model Integration:</p><p id="cfb7">We’ll integrate the Language Model functionality into our server to handle chat interactions. This may involve leveraging existing LM libraries or implementing custom logic to interact with the LM backend.</p><p id="7166">4. Error Handling:</p><p id="51e4">Error handling is crucial for ensuring the reliability of our server. We’ll design robust error handling mechanisms to gracefully handle errors and return meaningful error responses to clients.</p><p id="4375">5. Concurrency and Performance:</p><p id="2b52">Rust’s concurrency features will be leveraged to ensure our server can handle multiple requests concurrently without compromising performance or safety. We’ll design our server to efficiently utilize system resources and minimize latency.</p><p id="b4f6">6. Configuration and Deployment:</p><p id="ee30">We’ll design our server to be configurable and deployable in various environments. This involves defining configuration options for server settings such as port number, log levels, and any other relevant parameters.</p><p id="e1e2">7. Testing and Quality Assurance:</p><p id="0a99">Comprehensive testing will be an integral part of our design process. We’ll plan for unit tests, integration tests, and possibly end-to-end tests to ensure the reliability and correctness of our server implementation.</p><p id="647b">Conclusion:</p><p id="860d">This design brainstorming session provides a high-level overview of the key components and considerations involved in building a Rust-based REST server for serving Language Model requests. By carefully planning and designing our server architecture, we can create a robust and scalable platform for handling LM interactions effectively.</p></article></body>

Why Curation Is Less Important on Medium now

Make Peace with the new medium changes

Medium made big changes in the first week of October month. Medium Staff want medium as the more relational platform.

As the medium is social media site, Medium made some changes to cope with its social media tag. 20 thousand Stories post on medium daily. Medium has a staff of 85 members. Is it possible to read everyone’s story?

Medium’s new update in which medium said it wants to create more reader and writer relationships. Medium is implementing the same strategy, like other social media site use in their medium feed.

Why Medium is becoming like YouTube

Medium is trying to make its front page more like YouTube. When you open your YouTube, you get videos according to your watching history. You get videos of YouTuber who you watched his video’s irrespective you follow him or not.

Let consider YouTube channel as a publication on medium and we write as a content creator. On YouTube, when you follow YouTuber, you get his videos on his feed.

What it shows. Social media platforms are becoming more relational now. Now, on medium, you will get your feed related to what you read previously and what you follow. Medium has a strong algorithm. They will take care of it.

The medium front page you see faces of writer you follow and the latest content from them. It Means followers are more important for medium.

Let us take an example. If Jenny loves reading romance stories, she will follow romance story writers and publication. Some writer writes about the same topic on the medium daily. She will open the writer’s profile again and again, and unknowingly she will create a relationship with the writer.

Means a medium reader can tell their favourite writer's name. It is the same method that the medium has implemented as Quora. Quora has some popular writer and reader read their every post. We call this a social media relationship.

Creating a social media relationship will help medium to grow as reading and learning platform.

4 Things That Make Medium More Competitive For Writers

Medium is best platform for blogging but what makes medium more competitive read this…

medium.com

Why the writers should not worry about curation

I never worried about curation and now medium also doesn’t want curation as a hurdle between reader and writer. If you are writing quality and informative content for readers, then they will follow you.

Look for 1000 loyal followers. Rest will come behind. Write to them and they will follow you everywhere on social media. When you make loyal followers on the medium, you don’t need the support of curation. Medium algorithm peak your story when it gets more views. The medium algorithm is non-human, it doesn’t see curated and non-curated when it picks your story to see more readers. Distribution is human intervention.

New medium changes will help writers who struggle with curation. Now writers can write for their tribe.

But medium still continues to curate stories for further distribution. But start writing for your followers to make 1000 loyal one. You will see your graph increasing vertically.

21 Ways To Earn Money On Medium Without Getting Curated.

Medium help guide

medium.com

How recent changes will affect earning

Recent changes will affect top medium earners. as the medium changed their front page readers will read more from their following which can reduce the reading time of medium self-promoted stories.

But writers who have more than 1oK followers will need not to worry. But we can see a massive decline in medium earning of top earners. Medium earning will now become less fluctuating.

But for new writers, it is not good news. They have to struggle to gain a new follower. But consistency will help them with recent changes.

4 Strategies medium writer should apply now

Post daily

To keep up your followers engaging with your content, you need to post daily. So readers can see your face daily. If you're posting at least 3 times a week, it will help you with recent changes.

write on one topic

Readers will read the content more if you write about one specific topic. Reader read about the same topic again and again. It is basic human nature. We like what entertains us. Same as with medium readers.

Share on social media

Sharing content on social media helps you to get more exposure. You can share it everywhere. Many writers don’t share their work. Tell people to read your work. In this way, you can grow.

build email list

Start building an email list from day one. It is the best marketing strategy. It will help you build a relationship with your tribe and you can sell more products.

I see medium changes with positive aspects. Changes are inevitable. We should follow them.

Want to read more stories go here.