Automated Collection of Net Income Data from Big Companies — Small Python projects #2

Companies need money to finance their operations. To raise capital they can sell parts of their company. These parts are called shares and they are bought and sold on the stock market.

But before investors buy shares they might want to study the company’s finances. Luckily, companies that have their shares on stock markets release financial statements.

One type of financial statement is the Income Statement. It reports the company’s financial performance over a past period (year or quarter). An important part of an Income Statement is the net income. This is calculated as follows:

net income = revenue — expenses+gains-losses

If the net income is postive, the company made a profit. If it is negative the company made a loss.

As you might guess, the net income of a company is very important. It can be found on the Internet. But today we are going to write some software that automatically collects the net income data for us.

This is not financial advice or anything like that, I just want to show how you can use programming to automate processes.

This article is part of a series about small Python projects, you can find the previous episode here:

Data source: Alpha Vantage API

The source of our data will be the Alpha Vantage API. An API is basically a system that can receive messages and automatically respond to these incoming messages.

Alpha Vantage collects and shares a lot of financial data about companies, shares, (crypto)currencies and commodities. We will send their API a message specifying what data we need and, hopefully, their API will respond with the requested data.

To be able to communicate with the Alpha Vantage API you have to claim a free API key. The key that you receive has to be included in every Alpha Vantage API request you make.

API Documentation

For programmers to be able to find what kind of request they should make, API owners often provide documentation. In the Alpha Vantage API documentation we can find the Income Statement request.

Creating our Python code

The goal will be to write a function that returns the quarterly net incomes of the recent years.

First of all, APIs are used by sending requests to a certain URL. In the documentation we can find an example URL to get Income Statements:

https://www.alphavantage.co/query?function=INCOME_STATEMENT&symbol=IBM&apikey=demo

You might notice that they specify IBM, the symbol or ticker that the company IBM can be found at on the NYSE. This will make the API return IBM’s Income Statement data.

Since the API request is to get data from the API, we have to make a get-request. In Python you can make get-requests with the get function from the requests package. To learn more about Python packages and how to install them you can read this article:

You can use the get function by passing a URL to it.

import requests

key = 'secret'
company = 'F'

url = 'https://www.alphavantage.co/query?'
url += 'function=INCOME_STATEMENT'
url += '&symbol='+company
url += '&apikey='+key
r = requests.get(url)
data = r.json()

Instead of hardcoding IBM as a symbol we will insert variable company in the URL. We give company the value 'F', this is the symbol of the Ford company. We also insert our key which we store in variable key. To keep my key secret, I replaced it with string secret in this article.

We use the json method on response-object r to transform the JSON data that the API sends back into a dictionary.

We are looking for quarterly data and so we want to use the data stored at key quarterlyReports. This is an array/list of reports, here is an example of one quarterly report’s data:

{'fiscalDateEnding': '2022-09-30', 'reportedCurrency': 'USD',
 'grossProfit': '2622000000', 'totalRevenue': '39392000000',
 'costOfRevenue': '36770000000', 'costofGoodsAndServicesSold': '34354000000',
 'operatingIncome': '504000000', 'sellingGeneralAndAdministrative': '2847000000',
 'researchAndDevelopment': 'None', 'operatingExpenses': '2847000000',
 'investmentIncomeNet': '181000000', 'netInterestIncome': '-321000000',
 'interestIncome': '2947000000', 'interestExpense': '321000000',
 'nonInterestIncome': '39257000000', 'otherNonOperatingIncome': '104000000',
 'depreciation': '1895000000', 'depreciationAndAmortization': '2503000000',
 'incomeBeforeTax': '-1022000000', 'incomeTaxExpense': '-195000000',
 'interestAndDebtExpense': '456000000',
 'netIncomeFromContinuingOperations': '-930000000',
 'comprehensiveIncomeNetOfTax': '-1547000000', 'ebit': '504000000',
 'ebitda': '3558000000', 'netIncome': '-827000000'}

The data we are interested in can be found at the first and last key: fiscalDateEnding and netIncome.

We can loop over the list of quarterly reports and extract the data stored at the 2 above mentioned keys.

for report in data['quarterlyReports']:
    print(report['fiscalDateEnding'])
    print(report['netIncome'])
    break

The above code prints the date and net income of the first quarterly report:

2022-09-30
-827000000

Before we advance we will create 2 helper functions to transform the date-string into an integer that holds the year and and an integer that holds the quarter number:

def get_year(date_str_YYYY_MM_DD):
    return int(date_str_YYYY_MM_DD.split('-')[0])


def get_quarter(date_str_YYYY_MM_DD):
    month = int(date_str_YYYY_MM_DD.split('-')[1])
    return 1 if month==3 else 2 if month==6 else 3 if month==9 else 4

I chose to store the data in a dictionary that is structured like this:

quarterly_net_incomes = {2022:{4:10000, 3:20000, 2:-5000, 1:10000},
                         2021:{4:5000, 3:10000, 2:10000, 1:10000}
                        }

It is a nested dictionary with 2 depths. At the first level it is a dictionary that has a key-value pair for each year. The year number is the key and the value is a dictionary.

The second level consists of the dictionaries that are stored at each year number. These dictionaries contain key-value pairs for each quarter. The quarter number is the key and the value is the net income at that quarter.

To be able to easily make nested dictionaries, we use a defaultdict from the standard collections package. This is how that looks in practice:

from collections import defaultdict

quarterly_net_incomes = defaultdict(dict)
for report in data['quarterlyReports']:
    year = get_year(report['fiscalDateEnding'])
    quarter = get_quarter(report['fiscalDateEnding'])
    quarterly_net_incomes[year][quarter] = report['netIncome']

In this code we loop over the quarterly reports and store the net income in the nested dictionary quarterly_net_incomes.

Making a function from our Python code

Now we will transform all our code into a function that can be easily called and handed a company’s symbol. The function will return the nested dictionary containing the net income per quarter per year.

We will also include some statements to make sure no errors occur when no data is returned by the API.

import requests
from collections import defaultdict


def get_latest_quarterly_net_incomes(key,company):
    url = 'https://www.alphavantage.co/query?'
    url += 'function=INCOME_STATEMENT'
    url += '&symbol='+company
    url += '&apikey='+key
    r = requests.get(url)
    if r.status_code==200 and r.json():
        data = r.json()
        print(data)
        quarterly_net_incomes = defaultdict(dict)
        for report in data['quarterlyReports']:
            year = get_year(report['fiscalDateEnding'])
            quarter = get_quarter(report['fiscalDateEnding'])
            quarterly_net_incomes[year][quarter] = report['netIncome']

        return quarterly_net_incomes
    else:
        print('Failed getting income statements of ' + company)
        return {}

Checking whether the response’s status is 200 means that we check whether the API request was succesful. 200 as a response status code means ‘Success’!

And incoorporating r.json() in our if-statement equals checking whether any JSON data was actually sent. If no JSON data is sent r.json() will return an empty dictionary. And since empty dictionaries are false values, the if-clause won’t be executed in that case.

If the request was succesful and the dictionary is filled with data the quarterly_net_incomes dictionary is made, filled with data and returned.

If not, a message is printed as a notification and an empty dictionary is returned.

Testing our function

Now the function is finished we can of course call it. We have to pass our key and a company symbol.

We will test our function using a non-existing company first:

company = 'Fff'

net_incomes = get_latest_quarterly_net_incomes(key,company)
if net_incomes:
    print(net_incomes[2022][3])

    for year, year_data in net_incomes.items():
        print(year)
        print(year_data)

The output is:

Failed getting income statements of Fff

If we change company to F for Ford the output is:

-827000000
2022
{3: '-827000000', 2: '667000000', 1: '-3110000000'}
2021
{4: '12282000000', 3: '1832000000', 2: '561000000', 1: '3262000000'}
2020
{4: '-3810000000', 3: '2385000000', 2: '1117000000', 1: '-1993000000'}
2019
{4: '-2436000000', 3: '425000000', 2: '148000000', 1: '1146000000'}
2018
{4: '-17000000', 3: '991000000', 2: '1066000000', 1: '1736000000'}
2017
{4: '1889000000'}

With -827000000 being the net income from 2022 Q3 corresponding to: print(net_incomes[2022][3]. And the rest gets printed in the for-loop that follows.

This article is part of a series about small Python projects, you can find the next episode here:

Thank you for reading!

You can get full access to all my posts by joining Medium. Your membership fee directly supports me and other writers you read. You’ll also get full access to every story on Medium: