How to Extract Corporate Bonds Information on Markets Insider via Python?

Summary

This text provides a tutorial on how to extract bond information from Markets Insider's bonds finder using Python.

Abstract

The text begins by explaining the concept of corporate bonds and the structure of Markets Insider's bonds finder website. It then proceeds to provide a step-by-step guide on how to use Python to extract bond information from the website. The tutorial covers importing necessary packages, setting up a for-loop to iterate through pages, sending requests to the URL, parsing the data into a dataframe, and inserting the data into SQLite. The author notes that there is no detailed information like ISIN or Coupon Payment Date, which is located in the hyperlink of each item, and promises to discuss this in the next article.

Opinions

The author assumes the reader has a basic understanding of Python and SQLite.
The author emphasizes the importance of extracting bond information from Markets Insider's bonds finder.
The author acknowledges the limitations of the current method, which does not extract detailed information like ISIN or Coupon Payment Date.
The author encourages readers to support their work by buying them a coffee.
The author promises to provide more content in the future.
The author encourages readers to clap, follow, and sign up for their newsletter.
The author recommends an AI service that provides the same performance and functions as ChatGPT Plus(GPT-4) but is more cost-effective.

How to Extract Bonds Information on Markets Insider via Python? Part 1

In this series of articles, we will study how to use Python to extract the bond information in Markets Insider’s bonds finder.

First, let’s study on the structure of its websites before extracting the data.

https://markets.businessinsider.com/bonds/finder
?p=1 (page)
&borrower= (specific company)
&maturity= (shortterm: 0-3 yers / midterm: 3-10 years/ longterm: > 10 years)
&yield= (0: < 5% / 5:5%-10% / 10: 10%-20%)
&bondtype=(6%2c7%2c8%2c19: Corporate/ 2%2C3%2C4%2C16: Government, etc)
&coupon= (0: < 5% / 5:5%-10% / 10: >10%)
&currency= (333: USD / 534: JPY/ 846: SGD, etc)
&rating= (Moody's rating)
&country= (18: USA/ 33: China/ 27/ Singapore, etc)

Step 0: Import all related packages.

from datetime import date
from datetime import timedelta
import pandas as pd
import requests
import sqlite3
from google.colab import drive

drive.mount('/content/drive')
today = date.today()
con = sqlite3.connect('/content/drive/MyDrive/data/Stock.db')

Step 1: Define for-loop to loop through the pages. We are taking 0–3 years, yield between 5% to 10%

for i in range(1, 100):
  url = 'https://markets.businessinsider.com/bonds/finder?p='+ str(i) +'&borrower=&maturity=shortterm&yield=5&bondtype=6%2c7%2c8%2c19&coupon=0&currency=333&rating=&country=18'

Step 2: Send the request to the url and parse the data into dataframe.

res = requests.get(url)
res.encoding = 'big5'
html_df = pd.read_html(res.text)
df =  html_df [0]
df['As_Of'] = today

Step 3: Insert (or create) the data into SQLite.

df.to_sql('Coporate_Bond_Markets_Insider', con, if_exists='append')

You will find out there is no details like ISIN/ Coupon Payment Date, etc, which are actually located into the hyper link of each item. We will discuss it in the next article :) more to come!

Thank you. If you want to support Informula, you can buy us a coffee here :)

How to Extract Bonds Information on Markets Insider via Python? Part 1

In Plain English