Python Projects for Beginners a ten-Week Bootcamp Approach to Python Programming

drop_duplicates( )

Way too often will you see data with duplicate information, or just duplicate IDs. It’s imperative that you remove all duplicate records as it will skew your data, resulting in incorrect answers. You can remove duplicate records based on a single column or an entire record being identical. In our case, let’s remove duplicates based on similar names, which will remove the record we just added into our DataFrame:

# removing duplicates based on same names df = df.drop_duplicates( subset="names" ) df.head(10)

Go ahead and run the cell. This will remove the second record with the name “Jess.” By passing the column name into the subset parameter, we can remove all duplicates with the same name.
Note omitting the subset argument will remove only duplicate records that have identical values in all columns.

Pandas Joins

Often, you will have to combine data from several different sources to obtain the actual dataset you need for your exploration or modeling. Pandas draws heavily on SQL in its design for joins. This section assumes some knowledge of SQL and SQL joins. If you’re not familiar with how joins work in SQL, visit w3schools⁹ for reference material.
CHapter 10 INtroduCtIoN to data aNalYsIs

Creating a Second DataFrame

Let’s create a secondary DataFrame to represent our customers posting ratings about our company. We’ll create ratings for three users so we can see both inner joins and outer joins:

# creating another fake DataFrame to work with, having same names and a new ratings column ratings = {
"names" : [ "Jess", "Tyler", "Ted" ],
"ratings" : [ 10, 9, 6 ]
}
ratings = df.from_dict(ratings) ratings.head( )

Go ahead and run the cell. Now that we’ve created a second DataFrame, we can join the two DataFrames together, much like joining two tables together in SQL.