avatarJ3

Free AI web copilot to create summaries, insights and extended knowledge, download it at here

2885

Abstract

st’s advanced concurrency features provide excellent tooling for efficient and safe multi-threaded programming, maximizing your LLM’s throughput potential.</li><li><b>Web Ecosystem:</b> While Rust may be newer relative to languages like Python and JavaScript, its web development ecosystem is growing rapidly. Frameworks like Actix Web and Rocket offer mature solutions for building high-performance REST APIs.</li><li><b>Cross-Platform Compatibility:</b> Applications built with Rust can easily compile to run on virtually any operating system (Windows, Linux, macOS, etc.). This versatility is a tremendous advantage in deployment scenarios.</li></ol><h1 id="0df0">Let’s set the stage</h1><figure id="d2d7"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*AcpA4MkKboaPY0ONHHsN2g.jpeg"><figcaption></figcaption></figure><p id="4fe8">To interact with LLMs from Rust programs, there are a few primary methods:</p><ol><li><b>API Clients:</b> Many LLM services provide readily available REST APIs. Rust offers excellent HTTP client libraries, such as <code>reqwest</code>, to facilitate seamless communication with these APIs.</li><li><b>Model Hosting:</b> If you need low-latency or offline access, consider hosting language models directly within your Rust server. Rust bindings exist for popular frameworks like ONNX Runtime, allowing you to load and execute models locally.</li><li><b>Hybrid Approaches:</b> In some cases, a combination of the above approaches might be optimal. Your Rust server could interact with an external API when dealing with larger, more computationally intensive LLMs, while hosting smaller models locally for real-time tasks.</li></ol><h1 id="39fa">Our approach</h1><figure id="5b02"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*yxk1qTnQ9WfTTwKl-p20lg.jpeg"><figcaption></figcaption></figure><p id="c74d">In this design brainstorming session, we’ll outline the conceptual framework and key components for building a Rust-based REST server aimed at serving Language Model (LM) requests efficiently. Our goal is to design a scalable and performant server architecture that can handle various LM-related functionalities such as chat interactions, health checks, and version information retrieval.</p><h1 id="c75f">Problem Definition</h1><p id="8384"><b>Goal:</b> Establish a clear objective for our server. Possibilities include:</p><ul><li>Providing a central point of access and control for one or more large language models.</li><li>Offering an API layer for other applications to leverage LLM capabilities easily.</li><li>Abstracting away platform-specific LLM details behind a simple REST interface.</li></ul><h1 id="93e8">Target Users:</h1><p id="822d">Who are we building this server for?</p><ul><li>Developers building LLM-powered applications.</li><li>Data scientists conducting experiments with LLMs.</li><li>Int

Options

ernal services within an organization that need LLM functionality.</li></ul><h1 id="40ab">Design Thinking for a Rust LLM REST Server</h1><ol><li>Project Structure:</li></ol><p id="692a">We’ll start by defining the overall project structure, including modules, dependencies, and project organization. This involves setting up a Cargo-based project with appropriate dependencies for handling HTTP requests, JSON serialization, and any required LM-related functionality.</p><p id="08d8">2. Endpoint Design:</p><p id="347b">Next, we’ll design the REST API endpoints that our server will expose. Key endpoints may include:</p><ul><li><code>/api/query</code>: Endpoint for handling chat interactions with the Language Model.</li><li><code>/api/health</code>: Endpoint for performing health checks to ensure the server is running smoothly.</li><li><code>/api/app/version</code>: Endpoint for retrieving version information of the server application.</li></ul><p id="7e21">Each endpoint will have specific request/response formats and logic for handling incoming requests and generating appropriate responses.</p><p id="5660">3. Language Model Integration:</p><p id="cfb7">We’ll integrate the Language Model functionality into our server to handle chat interactions. This may involve leveraging existing LM libraries or implementing custom logic to interact with the LM backend.</p><p id="7166">4. Error Handling:</p><p id="51e4">Error handling is crucial for ensuring the reliability of our server. We’ll design robust error handling mechanisms to gracefully handle errors and return meaningful error responses to clients.</p><p id="4375">5. Concurrency and Performance:</p><p id="2b52">Rust’s concurrency features will be leveraged to ensure our server can handle multiple requests concurrently without compromising performance or safety. We’ll design our server to efficiently utilize system resources and minimize latency.</p><p id="b4f6">6. Configuration and Deployment:</p><p id="ee30">We’ll design our server to be configurable and deployable in various environments. This involves defining configuration options for server settings such as port number, log levels, and any other relevant parameters.</p><p id="e1e2">7. Testing and Quality Assurance:</p><p id="0a99">Comprehensive testing will be an integral part of our design process. We’ll plan for unit tests, integration tests, and possibly end-to-end tests to ensure the reliability and correctness of our server implementation.</p><p id="647b">Conclusion:</p><p id="860d">This design brainstorming session provides a high-level overview of the key components and considerations involved in building a Rust-based REST server for serving Language Model requests. By carefully planning and designing our server architecture, we can create a robust and scalable platform for handling LM interactions effectively.</p></article></body>

Python 4 Engineers — Keeping It In The Short-Term Memory

Test Yourself! Coding in Python, Again!— #PySeries#Episode 15

Short-term memory (STM) is the capacity to keep a small amount of information in mind in an active, readily available state for a short period of time.

This is the goal of this article. Keep python alive!

I enrolled myself in a program, supported by Huawei, and here is the test that I’ve got through to pass for my AI course.

Huawei is a leading global provider of information and communications technology (ICT) infrastructure and smart devices.

How about to Test Yourself?

Here we go!

Go to Google Colab and open a new notebook and answer these questions (or click bellow:):

Python 4 Engineers — Ep#15 — AI — Huawei.ipynb

01#PyEx — Python —List:

Which of the following matrices will throw an error in Python?
a) X = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
b) X = [[3, 3, 3] [4, 4, 4] [5, 5, 5]]
c) X = [(1, 2, 4), (5, 6, 7), (8, 9, 10)]
d) X = [2, 3, 4, 3, 3, 3, 4, 5, 6]

Hint: In Python, List is used to store collections of data.

Lists are created using square brackets.

Lists are separated by comma, so now it is easy to get a sense, right?

02#PyEx — Python — List — Range():

What will be the output of the following Python code?
A = [[1,2,3], [4,5,6], [7,8,9]]
[A[i][i] for i in range len(A))]
a) [1, 5, 9]
b) [3, 5, 7]
c) [4, 5, 6]
d) [2, 5, 8]

Hint: range() is a built-in function of Python. It is used when a user needs to perform an action for a specific number of times.

Note that the code snipped is inside a List.

03#PyEx — Python — Print():

The print function in Python 3 is enclosed in parentheses?
a) True
b) False

04#PyEx — Python — Functions:

Functions are organized code segments which cannot be reused to implement single or related operations?
a) True
b) False

Hint: Reusing code is key to building a maintainable system. And when it comes to reusing code in Python, it all starts and ends with the humble function. Take some lines of code, give them a name, and you’ve got a function (which can be reused).

05#PyEx — Python — Matrix:

Matrix A and matrix B can be added only if they have the same number of rows and columns?
a) True
b) False

Hint: In statistics two matrices may be added or subtracted only if they have the same dimension; that is, they must have the same number of rows and columns.

Addition or subtraction is accomplished by adding or subtracting corresponding elements. Is that true for python?

06#PyEx — Python — Matrix:

The determinant of a NxN matrix is a scalar?
a) True
b) False

Hint: For any N x M matrix, the determinant is a scalar value equal to the product of the main diagonal elements minus the product of it’s counter diagonal elements.

07#PyEx — Python — List:

Lists in Python can be delimited by square brackets [ ], with the first element at index 0?
a) True
b) False

08#PyEx — Python — Dictionary:

Which of the following format can define a dictionary in Python 3?
a) {“John”: ‘Paris’, “John”: ‘London’}
b) {“Jane”: ‘London’; “John”: ‘London’}
c) {“Jane”: ‘London’, “John”: ‘London’}
d) {“John”: ‘Paris’; “John”: ‘London’}

Hint: Dictionaries are written with curly brackets, and have keys and values.

As of Python version 3.7, dictionaries are ordered. In Python 3.6 and earlier, dictionaries are unordered.

The key repetition will consider only the last input.

09#PyEx — Python — Immutable Data Type in Python 3:

Which of the following is unchangeable in Python 3?
a) Tuple
b) Set
c) Array
d) List

Hint: Besides the different kind of brackets used to delimit them, the main difference between a tuple and a list is that the tuple object is immutable. Once we’ve declared the contents of a tuple, we can’t modify the contents of that tuple.

Some of the mutable data types in Python are list, dictionary, set and user-defined classes. On the other hand, some of the immutable data types are int, float, decimal, bool, string, tuple, and range.

10#PyEx — Python — Indexing in Python 3:

Which of the following statements about Python 3 index are true?
a) Default index from right to left is numbered from 0
b) Default index from right to left is numbered from -1
c) Default index from left to right is numbered from 1
d) Default index from left to right is numbered from 0

Hint: The list index starts with 0 in Python, from left to right

11#PyEx — Python — Indexing in Python 3:

What is the output of the following Python 3 code?
import numpy as np
 
A = np.matrix([[1,2],[3,4],[5,6]])
 
B = np.matrix([[12,11, 10],[9,8,7],[6,5,4], [3,2,1]])
 
C = np.matmul(B, A)
 
print(C.shape)
a) (2, 2)
b) (4, 4)
c) (3, 4)
d) (4, 2)

12#PyEx — Python — Functions in Python 3:

A three-digit integer number is called an Armstrong number if a sum of the cube of its digits equals the number itself. For example, 153 is an Armstrong number because 1 ^ 3 + 5 ^ 3 + 3 ^ 3 = 1 + 125 + 27 = 153. 
Mark the item that completes the IF statement in Python 3 code above and inform correctly if variable n is an Armstrong number.
a) n[0] ** 3 + n[1] ** 3 + n[2] ** 3
b) n[0] ^ 3 + n[1] ^ 3 + n[2] ^ 3
c) (n // 100) ** 3 + ( n // 10 % 10) ** 3 + (n % 10) ** 3
d) (n // 100) ** 3 + (n // 10) ** 3 + (n % 10) ** 3

13#PyEx — Python — Functions in Python 3:

During the first two years, a dog year is equal to 10.5 human years. After that, each dog year is equal to 4 human years. 
Given the following code above in Python 3, what correctly completes the last return statement?
a) dog_age * 4
b) dog_age * 4 + 21
c) (dog_age - 2) * 4
d) (dog_age - 2) * 4 + 21

14#PyEx — Python — Functions in Python 3:

What is the output from the following code in Python 3?
from functools import reduce
 
def abs(x):
  if x < 0:
    x = x * -1
  return x
 
print(reduce(lambda b, a : abs(a) - abs(b), [-5,-4,-3, -1]))
a) 1
b) -1
c) 15
d) -15

15#PyEx — Python — Functions in Python 3:

What will be the output of the following Python 3 code?
b = [2,3,4,5]
a = list(filter(lambda x:x%2, b))
print(a)
a) [2, 4]
b) []
c) [3, 5]
d) Invalid arguments for filter function

16#PyEx — Python — Functions in Python 3:

What will be output after we run the following Python 3 code?
vec = [0,1,2]
i = 30
while (i > 0):
  if i > 16:
    vec.append(i)
  elif i < 16:
    vec.pop()
  elif i == 16:
    vec.append(0)
  i-=1
print(len(vec))
a) 1
b) 0
c) 3
d) 2

17#PyEx — Python — Functions in Python 3:

What value of “val” will be printed after we run the following Python 3 code? 
vec  = [2,5,1,9,3,7]
val = 0
for idx, x in enumerate(vec):
  if val < x:
    val = vec[idx]
print(val)
a) 3
b) 9
c) 5
d) 7

18#PyEx — Python — Dictionary in Python 3:

Which of the following options is a Python dictionary?
a) {‘image_file’ = ‘./my_dir/imgs/img_258349xh3’, ‘image_len’ = img.shape[0] * img.shape[1]}
b) {‘id’: 7, ‘pass’: ‘password_key’, ‘width’: 250, ‘height’: 250}
c) (a: 7, b: ‘word’, c:numpy.arange(15))
d) [‘files’: ‘./my_files/tensorflow_files’, ‘folder_tam’: 15]

Answers

1-b; 2–a; 3-a; 4-b; 5-a; 6-a; 7-a;8-a & c;9-a; 10-d; 11-d; 12-c; 13-d; 14-b; 15-c; 16-c; 17-b; 18-b

Google Colab File

Down Raw File

Credits & References

INTRODUÇÃO A MACHINE LEARNING PARA CERTIFICAÇÃO HCIA-AI by crateus.ufc.br

https://www.huawei.com/en/

Posts Related:

00Episode#PySeries — Python — Jupiter Notebook Quick Start with VSCode — How to Set your Win10 Environment to use Jupiter Notebook

01Episode#PySeries — Python — Python 4 Engineers — Exercises! An overview of the Opportunities Offered by Python in Engineering!

02Episode#PySeries — Python — Geogebra Plus Linear Programming- We’ll Create a Geogebra program to help us with our linear programming

03Episode#PySeries — Python — Python 4 Engineers — More Exercises! — Another Round to Make Sure that Python is Really Amazing!

04Episode#PySeries — Python — Linear Regressions — The Basics — How to Understand Linear Regression Once and For All!

05Episode#PySeries — Python — NumPy Init & Python Review — A Crash Python Review & Initialization at NumPy lib.

06Episode#PySeries — Python — NumPy Arrays & Jupyter Notebook — Arithmetic Operations, Indexing & Slicing, and Conditional Selection w/ np arrays.

07Episode#PySeries — Python — Pandas — Intro & Series — What it is? How to use it?

08Episode#PySeries — Python — Pandas DataFrames — The primary Pandas data structure! It is a dict-like container for Series objects

09Episode#PySeries — Python — Python 4 Engineers — Even More Exercises! — More Practicing Coding Questions in Python!

10Episode#PySeries — Python — Pandas — Hierarchical Index & Cross-section — Open your Colab notebook and here are the follow-up exercises!

11Episode#PySeries — Python — Pandas — Missing Data — Let’s Continue the Python Exercises — Filling & Dropping Missing Data

12Episode#PySeries — Python — Pandas — Group By — Grouping large amounts of data and compute operations on these groups

13Episode#PySeries — Python — Pandas — Merging, Joining & Concatenations — Facilities For Easily Combining Together Series or DataFrame

14Episode#PySeries — Python — Pandas — Pandas Dataframe Examples: Column Operations

15Episode#PySeries — Python — Python 4 Engineers — Keeping It In The Short-Term Memory — Test Yourself! Coding in Python, Again! (this one:)

16Episode#PySeries — NumPy — NumPy Review, Again;) — Python Review Free Exercises

17Episode#PySeriesGenerators in Python — Python Review Free Hints

18Episode#PySeries — Pandas Review…Again;) — Python Review Free Exercise

19Episode#PySeriesMatlibPlot & Seaborn Python Libs — Reviewing theses Plotting & Statistics Packs

20Episode#PySeriesSeaborn Python Review — Reviewing theses Plotting & Statistics Packs

Python3
Notebook
Python 4 Engineers
Huawei
Google Colab
Recommended from ReadMedium