Use to_string() to Cease Python from Hiding the Physique of the Printed DataFrames | by Yufeng | Apr, 2023

3-Minutes Pandas

What ought to we do to see your entire printed dataframe after the execution of a Python script?

Generally operating via a Python script with out reporting any errors isn’t the one job of the debugging course of. We’d like to verify the capabilities are executed as anticipated. It’s a typical step within the exploratory knowledge evaluation to test how the info appears to be like like earlier than and after some particular knowledge processing.

So, we have to print out some knowledge frames or important variables in the course of the execution of the script, with the intention to test whether or not they’re “right”. Nonetheless, easy print command can solely present the highest and backside rows of the info body typically (as proven within the instance beneath), which makes the checking process unnecessarily exhausting.

Normally, the info frames are within the format of pandas.DataFrame, and should you use the print command instantly, you would possibly get one thing like this,

import pandas as pd
import numpy as np
knowledge = np.random.randn(5000, 5)
df = pd.DataFrame(knowledge, columns=['A', 'B', 'C', 'D', 'E'])
print(df.head(100))

You could have already seen that the center a part of the info body is hidden by three dots. What if we actually have to test what the highest 100 rows are? For instance, we need to test the results of a selected step in the course of a big Python script, with the intention to make certain the capabilities are executed as anticipated.

set_option()

Some of the easy options is to edit the default variety of rows that Pandas present,

pd.set_option('show.max_rows', 500)
print(df.head(100))

the place set_option is a technique that permits you to management the conduct of Pandas capabilities, which incorporates setting the utmost variety of rows or columns to show, as we did above. The primary argument show.max_rows is to regulate the utmost variety of rows to show and 500 is the worth we set as the utmost row quantity.

Despite the fact that this methodology is extensively used, it’s not ultimate to place it inside an executable Python file, particularly in case you have a number of knowledge frames to print and they’re desired to show completely different numbers of rows.

For instance, I’ve a script structured as proven,

## Code Block 1 ##
...
print(df1.head(20))
...
## Code Block 2 ##
...
print(df2.head(100))
...
## Code Block N ##
...
print(df_n)
...

now we have completely different numbers of prime rows to point out via your entire script, and typically we need to see your entire printed knowledge body, however typically we solely care in regards to the dimension and construction of the info body with out the necessity to see your entire knowledge.

In such a case, we most likely want to make use of the operate pd.set_option() to set the specified show or pd.reset_option() to make use of the default choices each time earlier than we print an information body, which makes it very messy and troublesome.

## Code Block 1 ##
...
pd.set_option('show.max_rows', 20)
print(df1.head(20))
...
## Code Block 2 ##
...
pd.set_option('show.max_rows', 100)
print(df2.head(100))
...
## Code Block N ##
...
pd.reset_option('show.max_rows')
print(df_n)
...

There’s truly a extra versatile and efficient means of exhibiting your entire knowledge body with out specifying the show choices for Pandas.

to_string()

to_string() instantly switch the pd.DataFrame object to a string object and after we print it out, it doesn’t care in regards to the show restrict from pandas .

pd.set_option('show.max_rows', 10)
print(df.head(100).to_string())

We will see above that although I set the utmost variety of rows to show as 10, to_string() helps us print your entire knowledge body of 100 rows.

The operate, to_string() , converts a whole knowledge body to the string format, so it might hold all of the values and indexes within the knowledge body within the printing step. Since set_option() is simply efficient on pandas objects, our printing string isn’t restricted by the utmost variety of rows to show set earlier.

So, the technique is that you simply don’t have to set something through set_option() and also you solely want to make use of to_string() to see your entire knowledge body. It is going to prevent from enthusiastic about which choice to set wherein half throughout the script.

Takeaways

1. Use set_option('show.max_rows') when you will have a constant variety of rows to show throughout your entire script.
2. Use to_string() if you wish to print out your entire Pandas knowledge body it doesn’t matter what Pandas choices have been set.

Thanks for studying! Hope you take pleasure in utilizing the Pandas trick in your work!

Please subscribe to my Medium if you wish to learn extra tales from me. And you too can be a part of the Medium membership by my referral hyperlink!