Free AI web copilot to create summaries, insights and extended knowledge, download it at here

Abstract

st’s advanced concurrency features provide excellent tooling for efficient and safe multi-threaded programming, maximizing your LLM’s throughput potential.</li><li><b>Web Ecosystem:</b> While Rust may be newer relative to languages like Python and JavaScript, its web development ecosystem is growing rapidly. Frameworks like Actix Web and Rocket offer mature solutions for building high-performance REST APIs.</li><li><b>Cross-Platform Compatibility:</b> Applications built with Rust can easily compile to run on virtually any operating system (Windows, Linux, macOS, etc.). This versatility is a tremendous advantage in deployment scenarios.</li></ol><h1 id="0df0">Let’s set the stage</h1><figure id="d2d7"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*AcpA4MkKboaPY0ONHHsN2g.jpeg"><figcaption></figcaption></figure><p id="4fe8">To interact with LLMs from Rust programs, there are a few primary methods:</p><ol><li><b>API Clients:</b> Many LLM services provide readily available REST APIs. Rust offers excellent HTTP client libraries, such as <code>reqwest</code>, to facilitate seamless communication with these APIs.</li><li><b>Model Hosting:</b> If you need low-latency or offline access, consider hosting language models directly within your Rust server. Rust bindings exist for popular frameworks like ONNX Runtime, allowing you to load and execute models locally.</li><li><b>Hybrid Approaches:</b> In some cases, a combination of the above approaches might be optimal. Your Rust server could interact with an external API when dealing with larger, more computationally intensive LLMs, while hosting smaller models locally for real-time tasks.</li></ol><h1 id="39fa">Our approach</h1><figure id="5b02"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*yxk1qTnQ9WfTTwKl-p20lg.jpeg"><figcaption></figcaption></figure><p id="c74d">In this design brainstorming session, we’ll outline the conceptual framework and key components for building a Rust-based REST server aimed at serving Language Model (LM) requests efficiently. Our goal is to design a scalable and performant server architecture that can handle various LM-related functionalities such as chat interactions, health checks, and version information retrieval.</p><h1 id="c75f">Problem Definition</h1><p id="8384"><b>Goal:</b> Establish a clear objective for our server. Possibilities include:</p><ul><li>Providing a central point of access and control for one or more large language models.</li><li>Offering an API layer for other applications to leverage LLM capabilities easily.</li><li>Abstracting away platform-specific LLM details behind a simple REST interface.</li></ul><h1 id="93e8">Target Users:</h1><p id="822d">Who are we building this server for?</p><ul><li>Developers building LLM-powered applications.</li><li>Data scientists conducting experiments with LLMs.</li><li>Int

Options

ernal services within an organization that need LLM functionality.</li></ul><h1 id="40ab">Design Thinking for a Rust LLM REST Server</h1><ol><li>Project Structure:</li></ol><p id="692a">We’ll start by defining the overall project structure, including modules, dependencies, and project organization. This involves setting up a Cargo-based project with appropriate dependencies for handling HTTP requests, JSON serialization, and any required LM-related functionality.</p><p id="08d8">2. Endpoint Design:</p><p id="347b">Next, we’ll design the REST API endpoints that our server will expose. Key endpoints may include:</p><ul><li><code>/api/query</code>: Endpoint for handling chat interactions with the Language Model.</li><li><code>/api/health</code>: Endpoint for performing health checks to ensure the server is running smoothly.</li><li><code>/api/app/version</code>: Endpoint for retrieving version information of the server application.</li></ul><p id="7e21">Each endpoint will have specific request/response formats and logic for handling incoming requests and generating appropriate responses.</p><p id="5660">3. Language Model Integration:</p><p id="cfb7">We’ll integrate the Language Model functionality into our server to handle chat interactions. This may involve leveraging existing LM libraries or implementing custom logic to interact with the LM backend.</p><p id="7166">4. Error Handling:</p><p id="51e4">Error handling is crucial for ensuring the reliability of our server. We’ll design robust error handling mechanisms to gracefully handle errors and return meaningful error responses to clients.</p><p id="4375">5. Concurrency and Performance:</p><p id="2b52">Rust’s concurrency features will be leveraged to ensure our server can handle multiple requests concurrently without compromising performance or safety. We’ll design our server to efficiently utilize system resources and minimize latency.</p><p id="b4f6">6. Configuration and Deployment:</p><p id="ee30">We’ll design our server to be configurable and deployable in various environments. This involves defining configuration options for server settings such as port number, log levels, and any other relevant parameters.</p><p id="e1e2">7. Testing and Quality Assurance:</p><p id="0a99">Comprehensive testing will be an integral part of our design process. We’ll plan for unit tests, integration tests, and possibly end-to-end tests to ensure the reliability and correctness of our server implementation.</p><p id="647b">Conclusion:</p><p id="860d">This design brainstorming session provides a high-level overview of the key components and considerations involved in building a Rust-based REST server for serving Language Model requests. By carefully planning and designing our server architecture, we can create a robust and scalable platform for handling LM interactions effectively.</p></article></body>

Why Black Girls Need Black Stories

When I used to teach, all of my students were Black, and the majority of them were girls. All of them were struggling readers, and a few had dyslexia.

I worked as a Reading Specialist with children ages 6–9 years old. I remember spending an entire academic year teaching a first grader the alphabet.

As most teachers can likely attest, even in the best of circumstances, keeping children motivated and focused on a task that specifically causes them frustration and anxiety is something of a Herculean feat. I spent over 40 hours each week trying to teach them the rules of a language that’s infamous for breaking its own guidelines.

Yet even beyond the resentment towards grammar, a lot of my kids weren’t interested in reading as an activity or stories as a form of enjoyment. One little girl in particular had nothing but impatience for the task. She was a third grader — I’ll call her Tiana. Her progress was the slowest of any of my students, and each day with her felt like someone was tap-dancing on my very last nerve.

The only thing that helped me keep my cool in the face of her constant attitude was knowing that it came from a place of self-consciousness. Reading was hard for her, and she didn’t think she’d ever be good at it. She knew she was behind the other students in her class — most of the students in her grade, honestly. It was disheartening for her, on top of being completely uninteresting. And I had no idea how to combat that. I had been provided with a small collection of books to fit with our curriculum, and I gave her as many different stories as I had access to — stories about heroes, princesses, spies, adventurers, dancers, regular kids doing regular things. She didn’t care about any of it.

Then one day I happened to bring one of my own books to read on my break.

When Tiana entered my classroom she stopped short, hovering at my side rather than making her way to her seat.

“What’s that?”

Her eyes were transfixed on the novel on my desk. I had a copy of Queenie by Candice Carty-Williams with me. I could immediately see that the image of black braids illustrated in such detail on the cover that had caught my eye in the bookstore was having the same effect on her. Braids piled high in an up-do…like the ones she currently had in her hair.

Around this time, I attended a lecture by Dr. Tonya Leslie, an educational consultant and children’s book author with over 20 years of experience in publishing. She gave a presentation entitled “Mirrors and Windows: Diversity in Children’s Literature.” Much of her work focuses on the need and methods that schools can use to incorporate more diverse and inclusive texts into their classrooms. Sitting in the audience listening to her speak immediately shed light on so much of what I was witnessing with my students.

For non-white children, the inability to find themselves reflected in stories has an immense impact:

“When children cannot find themselves reflected in the books they read, or when the images they see are distorted, negative, or laughable, they learn a powerful lesson about how they are devalued in the society of which they are a part.” (Source)

I imagine that for children with dyslexia and learning disabilities, these feelings are amplified. How dehumanizing is it to feel so invisible while you struggle?

I’d be lying if I said I didn’t shell out a few of my own bucks to get books in the class with beautiful little Black girls on the covers. I also made a habit of continuing to bring my own books to school with me. It didn’t stop Tiana from mixing up her B’s and D’s, but it gave her some motivation to want to improve. Just knowing that there were stories out there that she would find interesting and relatable, that would make her feel visible, helped take reading from a chore to a challenge.

I don’t teach anymore, but I hope whoever has taken my place has realized and understands this. I hope the school they work for believes and supports this. Our Black girls need Black stories — we have to make sure they get them.

Why Black Girls Need Black Stories

When I used to teach, all of my students were Black, and the majority of them were girls. All of them were struggling readers, and a few had dyslexia.

More on books and Blackness:

Afrofantasy and the Need for Black Escapism

Afrofantasy is the often mislabeled and misrepresented literary genre that we need right now.