Free AI web copilot to create summaries, insights and extended knowledge, download it at here

Abstract

st’s advanced concurrency features provide excellent tooling for efficient and safe multi-threaded programming, maximizing your LLM’s throughput potential.</li><li><b>Web Ecosystem:</b> While Rust may be newer relative to languages like Python and JavaScript, its web development ecosystem is growing rapidly. Frameworks like Actix Web and Rocket offer mature solutions for building high-performance REST APIs.</li><li><b>Cross-Platform Compatibility:</b> Applications built with Rust can easily compile to run on virtually any operating system (Windows, Linux, macOS, etc.). This versatility is a tremendous advantage in deployment scenarios.</li></ol><h1 id="0df0">Let’s set the stage</h1><figure id="d2d7"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*AcpA4MkKboaPY0ONHHsN2g.jpeg"><figcaption></figcaption></figure><p id="4fe8">To interact with LLMs from Rust programs, there are a few primary methods:</p><ol><li><b>API Clients:</b> Many LLM services provide readily available REST APIs. Rust offers excellent HTTP client libraries, such as <code>reqwest</code>, to facilitate seamless communication with these APIs.</li><li><b>Model Hosting:</b> If you need low-latency or offline access, consider hosting language models directly within your Rust server. Rust bindings exist for popular frameworks like ONNX Runtime, allowing you to load and execute models locally.</li><li><b>Hybrid Approaches:</b> In some cases, a combination of the above approaches might be optimal. Your Rust server could interact with an external API when dealing with larger, more computationally intensive LLMs, while hosting smaller models locally for real-time tasks.</li></ol><h1 id="39fa">Our approach</h1><figure id="5b02"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*yxk1qTnQ9WfTTwKl-p20lg.jpeg"><figcaption></figcaption></figure><p id="c74d">In this design brainstorming session, we’ll outline the conceptual framework and key components for building a Rust-based REST server aimed at serving Language Model (LM) requests efficiently. Our goal is to design a scalable and performant server architecture that can handle various LM-related functionalities such as chat interactions, health checks, and version information retrieval.</p><h1 id="c75f">Problem Definition</h1><p id="8384"><b>Goal:</b> Establish a clear objective for our server. Possibilities include:</p><ul><li>Providing a central point of access and control for one or more large language models.</li><li>Offering an API layer for other applications to leverage LLM capabilities easily.</li><li>Abstracting away platform-specific LLM details behind a simple REST interface.</li></ul><h1 id="93e8">Target Users:</h1><p id="822d">Who are we building this server for?</p><ul><li>Developers building LLM-powered applications.</li><li>Data scientists conducting experiments with LLMs.</li><li>Int

Options

ernal services within an organization that need LLM functionality.</li></ul><h1 id="40ab">Design Thinking for a Rust LLM REST Server</h1><ol><li>Project Structure:</li></ol><p id="692a">We’ll start by defining the overall project structure, including modules, dependencies, and project organization. This involves setting up a Cargo-based project with appropriate dependencies for handling HTTP requests, JSON serialization, and any required LM-related functionality.</p><p id="08d8">2. Endpoint Design:</p><p id="347b">Next, we’ll design the REST API endpoints that our server will expose. Key endpoints may include:</p><ul><li><code>/api/query</code>: Endpoint for handling chat interactions with the Language Model.</li><li><code>/api/health</code>: Endpoint for performing health checks to ensure the server is running smoothly.</li><li><code>/api/app/version</code>: Endpoint for retrieving version information of the server application.</li></ul><p id="7e21">Each endpoint will have specific request/response formats and logic for handling incoming requests and generating appropriate responses.</p><p id="5660">3. Language Model Integration:</p><p id="cfb7">We’ll integrate the Language Model functionality into our server to handle chat interactions. This may involve leveraging existing LM libraries or implementing custom logic to interact with the LM backend.</p><p id="7166">4. Error Handling:</p><p id="51e4">Error handling is crucial for ensuring the reliability of our server. We’ll design robust error handling mechanisms to gracefully handle errors and return meaningful error responses to clients.</p><p id="4375">5. Concurrency and Performance:</p><p id="2b52">Rust’s concurrency features will be leveraged to ensure our server can handle multiple requests concurrently without compromising performance or safety. We’ll design our server to efficiently utilize system resources and minimize latency.</p><p id="b4f6">6. Configuration and Deployment:</p><p id="ee30">We’ll design our server to be configurable and deployable in various environments. This involves defining configuration options for server settings such as port number, log levels, and any other relevant parameters.</p><p id="e1e2">7. Testing and Quality Assurance:</p><p id="0a99">Comprehensive testing will be an integral part of our design process. We’ll plan for unit tests, integration tests, and possibly end-to-end tests to ensure the reliability and correctness of our server implementation.</p><p id="647b">Conclusion:</p><p id="860d">This design brainstorming session provides a high-level overview of the key components and considerations involved in building a Rust-based REST server for serving Language Model requests. By carefully planning and designing our server architecture, we can create a robust and scalable platform for handling LM interactions effectively.</p></article></body>

What’s the Other Side of a Lie?

Illustration designed by **BSIENKART** *(used with permission from the artist)*

I have a question for all my friends who are journalists. What is the other side of a lie? I mean how does one go about reporting the other side of a lie? The reason I’m asking is that I don’t understand how a lie can have but one side. The lie can have no reality. A lie, by definition therefore, is not real. Yet out in Arizona there is a candidate for Governor who is running on a platform of non-reality or bald-faced lies, take your pick. She is saying that the last presidential election was stolen. Okay, I’ll bite. Who stole it exactly? And when and how was it stolen? Brandon didn’t steal it. He won the presidency, but he wasn’t the one who conjured up fake electoral college voters. Brandon has never accused the Tower Troll of nefariousness. The only guy bitching about the election is the Troll and his offspring. Here’s the part of all of this trolling and lying and fascist rhetoric that wonders me no end.

A lot of the Arizona voters are retirees and older folks who have a lot of experience under their hats. I mean, I get that folks of that age are sometimes set in their ways, but they are not stupid and they know perfectly well what reality is. I can not imagine that constituency buying into a tv newscaster’s lies. We know that news people do not report the news. They report the news that they can use to support their agenda or their point of view or the world which they have painstakingly constructed. But they can say the lie in a way that sounds like they believe it.

I suspect that it is not profitable to report the news unless you somewhat support the advertiser’s view of reality. The advertiser has carefully built a position which fills a need which is recognized in the economic community, after all, and he would naturally want to know about events which benefit the segment of a community whose need his product or service supports. For example, in a given city, let’s say there are two car dealerships. One sells Cadillacs and one sells Chevrolets. The dealership which sells Cadillacs may cater to a community which see themselves as better than working class people. The Chevrolet dealership may cater to more of a blue collar working class clientele simply because some Chevrolet models are more economic than any Cadillac models. In this example, we can see how the Chevrolet dealership clientele could tend to be liberal and favor governmental support for things they can not afford without help such as medical care in their old age. Whereas the Cadillac clientele would prefer to do it themselves because they can afford to do it themselves. Therefore the Cadillac clientele might not understand why everyone hasn’t done it themselves.

I can understand and appreciate the supporting reasons for either and both positions. What I do not understand is why or how reporting a not-real event such as a lie can be considered as accurate information for either conservative or liberal thought processes. Ms. Lake who is running for governor of Arizona is a prime example of what disturbs me. Ms. Lake is insisting that the last presidential election was stolen. Ms. Lake is not a stupid person so she knows that what she is saying can not be true. What she is saying has no basis in reality. I know that. You know that, and I just have to believe that anyone smart enough to report what Ms. Lake is saying knows that. Yet I keep hearing that Ms. Lake is still denying the reality of the election as opposed to those who say the election was on the up and up. I can see why Ms. Lake might lie to convince her disciples that she is a proper candidate but why in God’s name would a self-respecting reporter ever treat Ms. Lake as anything other than the scam artist that she is?

We’re all aware that reporting of current events are slanted to present the brand slant to the public. MSNBC has a liberal slant while FOX has a (supposedly) conservative slant. In other words, the two news organizations report current events in a way to favor the slant the organization uses to see the world. We could even call that slant a point from which they assemble the news of current events. The problem with that approach is that it does not take into account an exchange of information must be made from a neutral position to be treated as information. If the clientele of a news organization requires neutral news, neither organization provides it. I don’t know that neutrality is possible, but I do know that it is not necessary to report lies as though they represent one side of an issue. They don’t. A lie can not represent an issue at all.

We all know when someone is lying. But if they present their lie with confidence the press reports the lie as though it had substance. I suppose the press is being polite. There are some methods we all can use to determine if someone is stretching the truth.

If someone is being vague and offering few details we can pretty much assume there is a lack of reality in the statement. Have you ever heard a MAGA-ite state how the election was stolen? Or by whom or offer any proof of any kind? Any of that would be impossible if the election was not stolen, wouldn’t it?

Repeating questions before answering is another tell for a liar. Not many politicians repeat questions in public but that does not prove the truth of their statements.

Failing to provide specific details when a story is challenged is a pretty good indicator of a lie too. The MAGA-ites never offer details of how the election was stolen. They do not offer details about why they think that Brandon should be impeached either. They can not cite details about the nice people who invaded the capital, can not explain why people were armed that day or what they really wanted to accomplish by charging into the capital that day. One does have to wonder why people who were saying out loud that they wanted to hang Mike and for Nancy to come out are reticent to speak openly about their intentions. I’ve not heard many details about those kinds of statements.

There’s no reason to try to convince folks to look at a reality where they know they’re behaving badly for the purpose of admitting how badly they’re behaving. But I don’t think it’s necessary to be fair with someone who is behaving badly either. I do not see the need for protecting their feelings or to even give them any kind of credit for their bad behavior.

I mean we do not argue about who wins the World Series. The team with the most runs wins each game. The game can be manipulated but only by the people who enforce the rules. In baseball, should the umpires be unfair and manipulate the game through the application of the rules, we’d all call that cheating and no one would take the side of the umpires. We all know that guy who says the game was rigged, but at the bar next summer he’ll give you the batting average of the teams and tell you why the winner won and the loser lost. Can’t we do that same thing in politics?

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Make your vote count. The midterm elections nationwide are held this Tuesday, November 8. Visit vote.gov for the latest voter information.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Forging Agreement’s newest on-demand workshop is coming to Teachable November 15. Learn more about the ideas that John introduces in his blogs in the Program Theory On-Demand Workshop!

One way to think about how your mind works is to consider it as a biocomputer. It has an operating system and different types of applications and programs that run when called, in similar fashion to a regular computer. The results of this workshop will give you new ways to consider your own beliefs, emotions, feelings and logical thoughts and how you can choose to modify them to meet your own requirements.

The workshop combines a series of on-demand videos with group discussions. Program Theory is taught by John Worthington and Paul Grenci. Stay tuned for the launch November 15, and contact @forgingagreement on Facebook or Instagram, @forgeagreement on Twitter, or email [email protected] for more information!

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

The most recent online Reality Creation Through the Dyad Workshop has just wrapped up. The Dyad workshop is an exploration into the relationship that exists between two people- any two people. A Dyadic relationship can be with a spouse, significant other, business partner, co-workers, family members, etc. These relationships can be wonderful and challenging all at the same time, but that is the beauty of relationships; traversing the good, the bad and everything in between.

In this interactive workshop, couples/individuals are afforded the opportunity to examine ever so common problems and tried and true solutions that guarantee an ongoing relationship if one is warranted.

This workshop will be happening on a monthly basis and runs over 2 weeks of 4 consecutive evenings. Contact @forgingagreement on Facebook or Instagram, @forgeagreement on Twitter, or email [email protected] for more information!

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

If you would like to support John, please consider subscribing to Medium using his referral link. Due to Medium not supporting mobile-based referrals, please use a desktop browser to sign-up. This ensures that a portion of your commission goes straight to John to support his work on this blog. You will get unlimited access to all of John’s blogs, plus unlimited access to thousands of other writers. Thank you!

Join Medium with my referral link - John Worthington

Read every story from John Worthington (and thousands of other writers on Medium). Your membership fee directly…

medium.com