Free AI web copilot to create summaries, insights and extended knowledge, download it at here

Abstract

ime.datetime.now()), 'price': price, 'item': item, 'discount': discount }</pre></div>In our case, the output is:<div id="477f"><pre>{ 'date': '2018-05-13 13:37:21.414342', 'price': ' $169.99', 'item': 'Dyson AM08 Bladeless Pedestal Fan | White/Silver | Refurbished', 'discount': '$ 399.99 | 57% off' }</pre></div><h1 id="8e42">Step 2: Saving to S3</h1>To save our result to S3, we use <a href="https://boto3.readthedocs.io/en/latest/index.html">Boto 3</a>, the AWS SDK for Python. First we get a reference to S3. Then we create an <code>object</code> with given <code>bucket</code> and <code>file_name</code>(the bucket was created beforehand though this can be done programmatically). Finally, we convert our data to <code>json</code> and write our <code>data</code> into the object.<div id="acf2"><pre>import boto3 import json</pre></div><div id="34fd"><pre>def save_file_to_s3(bucket, file_name, data): s3 = boto3.resource('s3') obj = s3.Object(bucket, file_name) obj.put(Body=json.dumps(data))</pre></div>We will package this utility function in the handler file where we house the actual function Lambda will call. More on this in the following steps.Incidentally, if saving tabular data as in our example, we might choose a database instead of S3. We illustrate S3 here as it is also a good choice for documents, we are also frequently scraped items.<h1 id="dd59">Step 3: The Handler Function</h1>A Lambda function needs a <a href="https://docs.aws.amazon.com/lambda/latest/dg/python-programming-model-handler-types.html">handler function</a>, which is the function Lambda will execute when it gets called. We will put the handler function, along with our Python dependencies, in a sub-directory called <code>ebay_deal_scraper</code>. This will allow us to separate the files which will be part of the Lambda package, and ancillary project files such as the Serverless config file discussed in later steps.We name our handler function <code>scrape</code> and give it the signature required by Lambda. We don’t use the <code>event</code> or <code>context</code> parameters, but if you needed to pass data into your Lambda function, they are what you would use.<div id="2ee8"><pre>def scrape(event, context): data = deal_scrape() file_name = f"deals-{data['date']}" save_file_to_s3('ebay_daily_deals', file_name, data)</pre></div>Our handler calls our <code>deal_scrape()</code> function, then writes the returned data to S3 under a file name based on the date.<h1 id="6b1f">Step 4: Packaging our Function</h1>Our custom code is ready to go, but Lambda also requires you include your dependencies in the package you upload to AWS. In our case this means <code>pip installing</code> our Python packages locally. In the <code>ebay_deal_scrapper</code> directory, we run:<div id="bc42"><pre>pip3 install requests bs4 -t .</pre></div>(if you have any problems, see this <a href="https://stackoverflow.com/questions/24257803/distutilsoptionerror-must-supply-either-home-or-prefix-exec-prefix-not-both">Stack Overflow issue</a>)This will install the <code>requests</code> and <code>beatiful soup</code> packages in our directory. Lamdba has<code>boto3</code> pre-installed, so you don’t need to include it.Incidentally, including dependencies can be quite hairy if you require platform-dependent C/C++ libraries like <a href="https://www.boost.org/">Boost</a>, and may require you to use Docker to bundle everything together under the Amazon Linux OS that Lambda requires. But that’s the topic of another blog post.When you have dependencies, you probably want to use a zip file to package everything up. We give ours the generic name of <code>package.zip</code>:<div id="f3ce"><pre>zip -r package.zip *</pre></div><h1 id="b305">Step 5: Deploying to Lambda</h1>We will use the Serverless framework to deploy to AWS. Serverless offers a set of command line tools which make it very easy to deploy to the major serverless cloud providers including AWS. You can install it using <code>npm</code>:<code>npm install -g serverless</code>You specify Serverless deployment instructions in a file called <code>serverless.yml</code>. If you want to generate a boilerplate file, you can run:<div id="4295"><pre>serverless create --template aws-python</pre></div>This will also create a boilerplate<code>handler.py</code> file, but all we need there is the function which we specify i

Options

n the <code>serverless.yml</code> config file under the <code>functions</code> section.<div id="7831"><pre>service: ebay-deal-scraper provider: name: aws runtime: python3.6 package: artifact: ebay_deal_scraper/package.zip functions: ebay_scrape: handler: handler.scrape</pre></div>The <code>provider</code> section tells Serverless we’re deploying to AWS and that we’re using <code>python 3.6</code>.The <code>package</code> section is where we specify the zip file we created in the last step.We are ready to deploy!From the top directory of your project run:<div id="d1ed"><pre>serverless deploy</pre></div>If you have multiple AWS profiles (such as work and personal), you can specify a profile:<div id="981f"><pre>serverless deploy --aws-profile profile_i_want_to_use</pre></div>If all goes well, you should get the following output:<div id="4f5e"><pre>➜ ebay-deals-scrape git:(master) ✗ sd --aws-profile michael Serverless: Packaging service... Serverless: Uploading CloudFormation file to S3... Serverless: Uploading artifacts... Serverless: Validating template... Serverless: Updating Stack... Serverless: Checking Stack update progress... ......... Serverless: Stack update finished... Service Information service: ebay-deal-scraper stage: dev region: us-east-1 stack: ebay-deal-scraper-dev api keys: None endpoints: None functions: ebay_scrape: ebay-deal-scraper-dev-ebay_scrape Serverless: Removing old service versions...</pre></div>You can now try to invoke your function with:<div id="b911"><pre>serverless invoke -f ebay_scrape # --aws_profile profile_name </pre></div>This will result in an AccessDenied error. To fix this, you need to add permission to your Lambda function’s role (this gets created together with the function).<h1 id="da14">Step 6: Giving Lambda S3 Privileges</h1>Go to the Roles page in the IAM section of the AWS dashboard. Find the role for your Lambda function, in our case <code>ebay-deal-scraper-dev-us-east-1-lambdaRole</code> and add a policy which allows it to access S3. <code>AmazonS3FullAccess</code> will work for our demo, though in production you may want to create a new policy which is more restrictive.Once your function is deployed you should also be able to view and test it from the AWS console.<figure id="dc68"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*A1uCWEB2Hb-YS7lMEiXCww.png"><figcaption></figcaption></figure><h1 id="6080">Step 7: Scheduling the Lambda Function Using CloudWatch</h1>Final step! Go to the CloudWatch Management Page and click the<code>Rules</code> tab. Under Event Source, select <code>Schedule</code> and fill in a cron expression. We set ours to run every day at 6PM GMT, or afternoon in US Eastern Time. Next, in the <code>Targets</code> section, choose <code>Lambda function</code> in the select and then your Lambda function from the list in the <code>Function</code> select. You’re done!<figure id="1595"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*amgaVqAlsAl0pQdYaAP_PA.png"><figcaption></figcaption></figure>That’s the whirlwind tour of scraping the serverless way. I hope this gives a taste of the technologies involved, though each of them could be the subject of many, many blog posts.As far as scraping goes, with the rise of big data/machine learning, data acquisition is becoming more and more important. And if, for instance, you wanted to do something like feed your machine learning model with data to <a href="https://www.dataquest.io/blog/machine-learning-tutorial/">make price predictions for AirBnB</a>, scraping might be your only option. The serverless architecture exemplified here offers an efficient way to do this.</article></body>

Balboa

Yo, Adrian, what’s the exchange rate today?

Today’s New York Times Spelling Bee letters:

A, B, I, M, O, X, and center L (all words must include L).

Merriam-Webster says…

Silly little dictionary! Don’t you know balboa can’t possibly be a word if the New York Times says it ain’t?

For further fascinating facts, check out the Spelling Bee Master.

What’s your favorite g.n.a.w. from today’s puzzle?

My Two Cents

I’ve been waiting for this one for a couple of years now, maybe more. Ever since I started playing the Spelling Bee game, the word balboa has come up as an option with any combination of seven letters that includes A, B, L, and O (and with one of the four in the middle, of course).

It was a valid word the first time I saw it, and it still wasn’t a valid word today.

Perhaps this column might change that. (Yeah, right…)

Dollar is accepted in any puzzle that contains its letters. So is peso, if I remember correctly. So why is balboa not?

Balboa is a currency, just like peso and dollar. It appears in the dictionary and is spelled in lowercase with no initial capital letter.

What gives, Sam Ezersky editors? You really need to explain this one to me.

To honor an a-hole

Vasco Núñez de Balboa was a Spanish explorer, governor, and conquistador. His claim to fame is crossing the Isthmus of Panama (that skinny part where the canal now lies) and becoming the first European to see the Pacific Ocean. And of course, getting his profile stamped on Panamanian currency around 400 years later.

He arrived in the New World in 1500 and first settled on the island of Hispaniola, today know as the piece of land Haiti and the Dominican Republic share. He went to Colombia 10 years later and founded the settlement of Santa María la Antigua del Darién, the first permanent European settlement on the mainland of either of the Americas.

This involved, as it usually did back then, killing a bunch of natives. Balboa earned the respect of his peers and ended up getting rid of the current mayor and declaring himself the new boss of the city. Not content with that, he also kicked out a governor who challenged him later on, and declared himself the new governor.

Umm… there seems to be a pattern here, right? It gets worse.

The Italian historian Peter Martyr d’Anghiera wrote that Balboa got upset with some of the indigenous people whom he thought were too “effeminate”. As a result, he fed forty of them to his dogs.

His cruelty and sadism continued unabated until a crueler and more sadist guy named Pedrarias Dávila tricked Balboa, arrested and tried him, and had him beheaded.

Still not enough to make up for feeding people who dressed “too girly” to a bunch of dogs, in my opinion.

So, how did Panama chose to address this guys brutal history? Simple. The country decided to name their national currency after him.

When Panama became independent from Colombia, they decided to replace the peso with a new coin in 1904. They tied the balboa to the U.S. dollar at an exchange rate of 1:1, which is how it has operated for most of the 117 years since.

The balboa coins have been issued as coins of 1 balboa; 50 céntesimos (hundredths); 25 centésimos; 10 centésimos; 5 centésimos; and, interestingly, 2 1/2 and 1/2 centésimos. Yeah, two and one-half cents. One day I’ll write about the Venezuelan locha, a coin worth 12.5 cents that existed when I was a kid.

It was only in 1941 that the government started issuing balboas in paper money.

The seventh greatest

The other balboa that’s been swirling around in your mind since you read the title of this article is, of course, soccer star Marcelo Balboa. No?

How about Rocky Balboa, voted the 7th greatest movie hero by the American Film Institute when they came out with the 100 Years… 100 Heroes and Villains list in 2003.

“Wait a second!” you cry out indignantly. “There are six movie heroes greater than Rocky Balboa? How is that possible?”

It’s true. I’m sorry. To see who they are, read through to the end. Or you could just skip the rest of this article, scroll down, and find out. If you do, please promise me you’ll scroll really slowly.

How incredible has Rocky Balboa’s fictional life been? Well…

In the 1980s the Rocky statue that Stallone commissioned for the third installment of the series found a permanent home at the steps of The Philadelphia Museum of Art.
In 2013, a poll of former heavyweight champions and boxing writers ranked Balboa as the best boxer among all the fighters in the film series.
In 2014, Rocky Balboa was the Inaugural Induction to the Fictitious Athlete Hall of Fame.

To top it off, his achievements even rubbed off on his creator, Sylvester Stallone. In 2011, the actor was inducted into the International Boxing Hall of Fame for his work on the films.

Maybe Panama should change their currency to honor this Balboa. They don’t even need to switch names.

What currency, you ask? The one that the editors of the New York Times Spelling Bee saw and said: “Gee, Not A Word”.

Okay, as promised, here they are. The six fictional heroes who beat out Rocky Balboa in the poll (with the movies in parentheses).

Atticus Finch (To Kill a Mockingbird)
Indiana Jones (Raider of the Lost Ark and other movies series)
James Bond (umm, just the MC of a bunch of spy movies in the past six decades and counting)
Rick Blain (Casablanca). I know what you’re thinking: “Play it again, Sam”. Blain never said it.
Marshall Will Kane (High Noon). This one I found surprising.
Clarice Starling (The Silence of the Lambs). I know what you’re thinking: “Hello, Clarice”. Lecter never said it.

Now that I’ve rewarded you, please check out my previous entry on words that g.n.a.w. at you:

Woad

A primer on a primary color

medium.com