Computer Science Hub

Pandas is a Python-based library which can be used for Data Manipulation and Analysis. Nowadays a number of companies and even small businesses are collecting tonne of data literally everyday, this has happened because of lowering of costs for storing Data specifically in Clouds-like AWS or Microsoft Azure. But the next question for Businesses to ask is, how to use that data for business decision making? Otherwise, there is no use of paying to Cloud Companies for collecting data. So for drawing decisions from collected data and using those decisions for either improving Business Decision Making or for improving the products which Business is offering. So in order to improve Business Decision Making or Improving Products, data already collected need to be well analysed. For doing businesses are continuously hiring people who can help for analyzing data. For doing this analysis of data, there are a lot of tools available in the Software Market, Pandas Python Library is one of these. Here in this article, I've collected together some commonly asked Pandas-related questions in the Job Interviews. Just to give some context around why learning Pandas is important, I just did a search on Linkedin for the keyword "Pandas" under Job Search and filtered down results for "Past Month" and "United States". It popped up with "2701 results" meaning there were atleast 2701 jobs posted on Linkedin in USA in last month. From this it can be estimated how many Businesses are looking for people who know "Pandas Python Library". Similar is true for other countries like Canada, Australia, India etc. So in this day and age, its crucial to learn how to use "Pandas Python Library" for getting a job role like Data Engineer or Data Scienctist. Q. 1 - What is Pandas Python Library? Pandas is a Python Library that helps in the easier representation of data in memory to perform analysis. Pandas help in faster representation and processing of data. Q. 2 - How does Pandas represent data? Pandas data representation is in similar line to an excel sheet which consists of row and columns. Columns in Pandas are known asSeriesThe collection of series is calledData Frame Q. 3 - How to create Series in Pandas? Q. 4 - How to create Data Frame in Pandas? Data Frame in Pandas can be created either directly from a dictionary or by combining various series. import pandas as pd country_population = {'India': 1100000, 'China': 45679000, 'USA': 3400000} population = pd.Series(country_population) #print(population) country_land = {'India': '2000 hectares', 'China': '4000 hectares', 'USA': '3000 hectares'} area = pd.Series(country_land) #print(area) df = pd.DataFrame({'Population': population, 'SpaceOccupied': area}) print(df) Output of Above Code Q. 5 - How are missing values represented in Pandas DataFrame? In Pandas DataFrame missing values are represented as NaN import pandas as pd missing = pd.DataFrame([{'a': 1, 'b': 2}, {'b': 3, 'c': 4}]) # {'a': 1, 'b': 2} and {'b': 3, 'c': 4} # Value of c is not in first dictionary, but value of a is not in second dictionary print(missing) Output of Above Code Q. 6 - Explain the process of creating indexes in pandas? Indexes can be created using Pandas' Index function. Indexes support intersection and union. import pandas as pd index_A = pd.Index([1, 3, 5, 7, 9]) index_B = pd.Index([2, 3, 5, 7, 11]) Q. 7 - Explain various attributes associated with Pandas Series. Pandas Series attributeDescriptionSeries.axesStands for rowSeries.dtypeData type of the object is given by this attributeSeries.emptyCheck if Series is emptySeries.ndimDimensions of data are given backSeries.sizeSize or number of elements from data are given Series.valuesGets the values in the form of ndarraySeries.head()First n rows are returnedSeries.tail()Last n rows are returned Q. 8 - Explain various statistical measures supported by Pandas. Statistical Measure in PandasDescriptionaxesPrint row index as well as column indexsumCalculates sum of all seriesmeanCalculates mean of all seriesmedianCalculates median of all seriesstdCalculates standard deviationcountCalculates sum of various seriescumsumCalculates cumulative sum Q. 9 - Explain reindexing in Pandas. Reindexing allows us to modify the index of one data frame by keeping the other data frame as a reference. Q. 10 - Explain bfill and ffill. While reindexing NaN can be introduced .bfill and .ffill are used to handle NaN bfill - Fills the value from ahead value into the previous NaN valueffill - Fills the value from behind value into the missing NaN value Reindexing without using any method(bfill or ffill) import pandas as pd import numpy as np df1 = pd.DataFrame(np.random.randn(4, 3), columns=['col1', 'col2', 'col3']) df2 = pd.DataFrame(np.random.randn(2, 3), columns=['col1', 'col2', 'col3']) print(df2.reindex_like(df1)) Output of Above Code Reindexing with using methods(bfill or ffill) import pandas as pd import numpy as np df1 = pd.DataFrame(np.random.randn(4, 3), columns=['col1', 'col2', 'col3']) df2 = pd.DataFrame(np.random.randn(2, 3), columns=['col1', 'col2', 'col3']) print(df2.reindex_like(df1, method='ffill')) # Or method='bfill' Output of above code if method='ffill' Output of above code if method='bfill' Q. 11 - What all type of iterations are provided in Pandas Data Frame? Iterator for Pandas Data FrameDescriptioniteritems()To iterate over the (key, value) pairsiterrows()Iterate over the rows as (index, series) pairsitertuples()Iterate over the rows as namedtuples Q. 12 - How to sort column of Data Frame in Pandas? sort_index - Allows sorting based rowwise or column wisesort_values - Allows sorting based on values in a column Creating a Sample DataFrame import pandas as pd d = {'col1': [10, 93, 16, 23, 81, 283, 10], 'col2': [19, 145, 195, 1952, 785, 543, 83782]} df = pd.DataFrame(data=d) Output of Above Code Using sort_index for Sorting Sample Data Frame sorted_df = df.sort_index(ascending=False) print(sorted_df) Output of Above Code Using sort_values for Sorting Sample Data Frame sorted_df = df.sort_values(by='col1') print(sorted_df) Output of Above Code Q. 13 - How to override default reload option in Pandas? Q. 14 - Explain various DataFrame slicing options available in Pandas? .loc() - Slicing DataFrame based upon Label.iloc() - Slicing DataFrame based on Interger.ix() - Slicing DataFrame based on both Label and Integer Q. 15 - How can we handle NaN values in Pandas DataFrame? NaN values in a Pandas DataFrame can be handled in the following three ways: - dropna - Removing all the rows in DataFrame for which values in column are NaNpad - Replacing NaN values with previous non NaN values meaning replacing NaN with value just above it in same columnbackfill - Replacing NaN values with ahead non NaN values meaning replacing NaN with value just below it in same column Q. 16 - Explain "group by" function in Pandas? group_by allows to group data in a DataFrame based on single or multiple columns. Q. 17 - Explaing "merge function" in Pandas? Data Frame in Pandas support merge operations in which two related data from diverse data frames can be brought in a single view.There are different ways through which different DataFrames can be merged together. Below are some of these ways: - Merging using a column as id So if we have two dataframes let's say - df1 and df2 having data as in following table. Then merging by 'Name' will create a new DataFrame containing all rows for Name which are in df2 but not in df1.See below tables, these will make scenario little bit clearer. df1 IndexNameAge0Bob241John342Garry183Smith26 df2 IndexNameBirth Place0BobAustin1JohnMiami2GarryNew York import pandas as pd merged_dataframe = pd.merge(df1, df2, on='Name') print(merged_dataframe) Output of Above Code IndexNameAgeBirth Place0Bob24Austin1John34Miami2Garry18New York Doing a Left Merger of DataFrames In Left Merger, all data from left side will come and only those matching from right would come.Below is the code for doing Left Merger of two dataframes - df1, df2. (See picture just below code to better understand how Left Merger works in Pandas) import pandas as pd merged_dataframe = pd.merge(df1, df2, on='Name', how='left') # df1 being left side # df2 being right side print(merged_dataframe) Doing a Right Merger of DataFrames In right merge everything from right side comes and only matching in left would come else it would come as NaN. Below is the code for doing Right Merger of two dataframes - df1, df2. (See picture just below code to better understand how Right Merger works in Pandas) import pandas as pd merged_dataframe = pd.merge(df1, df2, on='Name', how='right') # df1 being left side # df2 being right side print(merged_dataframe) Doing an Outer Merger of DataFrames Data from both left and right DataFrames will come together and all non-existing values will be replaced by NaN. Below is the code for doing Outer Merger of two dataframes - df1, df2. (See picture just below code to better understand how Outer Merger works in Pandas) Q. 18 - Explain Pandas's concat method? concat method can be used for combining two different data frames either at row level or column level. For combining rows, just putting one DataFrame on top of another - pd.concat([Top DataFrame, Bottom DataFrame]) can be used.For combining columns, just putting one DataFrame on right side of another - pd.concat([Left DataFrame, Right DataFrame], axis=1) can be used. For better understanding Pandas's concat method let's have a look at two examples. Suppose that we have two DataFrames df1, df2 which contain following data. df1 IndexNameAge0Bob241John342Garry183Smith26 df2 IndexNameBirth Place0BobAustin1JohnMiami2GarryNew York Putting one DataFrame on Top of another - pd.concat([Top DataFrame, Bottom DataFrame]) print(pd.concat([df1, df2])) Output of Above Code IndexNameAgeBirth Place0Bob24.0NaN1John34.0NaN2Garry18.0NaN3Smith26.0NaN0BobNaNAustin1JohnNaNMiami2GarryNaNNew York Putting one DataFrame on Left side of another - pd.concat([Top DataFrame, Bottom DataFrame], axis=1) print(pd.concat([df1, df2], axis=1)) Output of Above Code IndexNameAgeNameBirth Place0Bob24BobAustin1John34JohnMiami2Garry18GarryNew York3Smith26NaNNaN Q. 19 - Explain all Data File Types which either can be read or written by Pandas? Data File TypePandas Function For ReadingPandas Function For WritingCSVread_csvto_csvJSONread_jsonto_jsonHTMLread_htmlto_htmlMS Excelread_excelto_excelHDF5 Formatread_hdfto_hdfFeather Formatread_featherto_featherParquet Formatread_parquetto_parquetMsgpackread_msgpackto_msgpackStataread_statato_stataSASread_sasNo Function for thisPython Pickle Formatread_pickleto_pickleSQLread_sqlto_sqlGoogle Big Queryread_gbqto_gbq Q. 20 - Compare functions in Pandas and R? Filtering, Sampling, Querying Functions in Pandas versus R RPandasdim(dataframe)dataframe.shapehead(dataframe)dataframe.head()slice(dataframe, 1:100)dataframe.iloc[:99]filter(dataframe, column1 == 1, column2 == 1)dataframe.query('column1 == 1 & column2 == 1')select(dataframe, column1, column2)dataframe[['column1', 'column2']]select(dataframe, column1:column3)dataframe.loc[:, 'column1':'column3']distinct(select(dataframe. column1))dataframe[['column1']].drop_duplicates()sample_n(dataframe, 10)dataframe.sample(n=10)sample_frac(dataframe, 0.01)dataframe.sample(frac=0.01) Sorting Functions in Pandas versus R RPandasarrange(dataframe, column1, column2)dataframe.sort_values(['column1', 'column2'])arrange(dataframe, desc(column1))dataframe.sort_values('column1', ascending=False) Transforming Functions in Pandas versus R RPandasselect(dataframe, col_one = column1)dataframe.rename(columns={'column1':'col_one'})['col_one']rename(dataframe, col_one = column1)dataframe.rename(columns={'column1':'col_one'})mutate(dataframe, c = a - b)dataframe.assign(c = dataframe.a-dataframe.b) Aggregate/Grouping Functions in Pandas versus R RPandassummary(dataframe)dataframe.describe()gdatafram <- group_by(dataframe, column1)gdataframe = dataframe.groupby('column1')summarise(gdataframe, avg=mean(column1, na.rm=TRUE))dataframe.groupby('column1').agg({'column1':'mean'})summarise(gdataframe, total =- sum(column1))dataframe.groupby('column1').sum()

Recent Posts