Table of Content:

- About Regularization
- Types of Penalties
- Regularization Technique
- Lasso Regression
- Example Of Lasso Regression
- Conclusion

## About Regularization

To understand Lasso Regression, first we have to know what is Regularization.

So, Regularization is the concept that is used to avoid over fitting of the data by adding penalty to achieve less variance with the test data. The model is likely to perform better at prediction thereafter.

In simple term, it reduces parameters and simplifies the model so that it has the lowest over fitting or say less variance with test data, hence better prediction with test data.

For example, when we are training the model with train dataset which have two nodes, it fitted well with a straight line and for this its residual become zero. But when we test the model with test dataset we get high variance, which means our model is over fitted and we have to remove it.

It is observed that when we train our model it gives 100 percent accuracy on train data, but when we test it with test dataset its accuracy is substantially lower.

## Types of Penalties

Regularization works by biasing data equal to or nearly equal to zero. In simple words it shrinks the slope of the data to find the best fit. Note the two Types of penalties as below.

### L1 regularization

It adds a L1 penalty to the absolute or mode value of the magnitude of coefficient. In this process some coefficient can become zero and eliminated from model.

### L2 regularization

It adds a L2 penalty to the square of the magnitude of coefficient. In this, all coefficients will shrink by the same factor to find the best fit for the model.

## Regularization Technique

There are two main regularization techniques:

- Ridge Regression
- Lasso Regression.

Both have different way of assigning a penalty to the coefficients. In this article, we will learn about Lasso Regularization technique.

## Lasso Regression

“LASSO” stands for Least Absolute Shrinkage and Selection Operator. This model uses shrinkage. Shrinkage basically means that the data points are recalibrated by adding a penalty so as to shrink the coefficients to zero if they are not substantial.

It uses L1 regularization penalty technique. This particular type of regression is well-suited for models showing high levels of multicollinearity or when we have to automate certain parts of model selection, like parameter elimination or feature selection.

We can represent Lasso loss functions mathematically as:

__Figure 1 : Mathematical Formulation for LASSO Loss Function__

## Example Of Lasso Regression

importnumpyasnpimportpandasaspdfromsklearn.model_selectionimporttrain_test_splitfromsklearn.preprocessingimportStandardScalerfromsklearn.linear_modelimportLassofromsklearn.metricsimportr2_score as ac

Load the data set by clicking on this link - **Vehicle Price dataset**

price_data = pd.read_csv('vehicle_price.csv') price_data.head(5) Unnamed: 0 Name ... New_Price Price 0 0 Maruti Wagon R LXI CNG ... NaN 1.75 1 1 Hyundai Creta 1.6 CRDi SX Option ... NaN 12.50 2 2 Honda Jazz V ... 8.61 Lakh 4.50 3 3 Maruti Ertiga VDI ... NaN 6.00 4 4 Audi A4 New 2.0 TDI Multitronic ... NaN 17.74

```
price_data.info()
RangeIndex: 6019 entries, 0 to 6018
Data columns (total 14 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 Unnamed: 0 6019 non-null int64
1 Name 6019 non-null object
2 Location 6019 non-null object
3 Year 6019 non-null int64
4 Kilometers_Driven 6019 non-null int64
5 Fuel_Type 6019 non-null object
6 Transmission 6019 non-null object
7 Owner_Type 6019 non-null object
8 Mileage 6017 non-null object
9 Engine 5983 non-null object
10 Power 5983 non-null object
11 Seats 5977 non-null float64
12 New_Price 824 non-null object
13 Price 6019 non-null float64
dtypes: float64(2), int64(3), object(9)
memory usage: 658.5+ KB
```

```
price_data.isnull().sum()
Unnamed: 0 0
Name 0
Location 0
Year 0
Kilometers_Driven 0
Fuel_Type 0
Transmission 0
Owner_Type 0
Mileage 2
Engine 36
Power 36
Seats 42
New_Price 5195
Price 0
dtype: int64
```

# Droping location, new price and unnamed column price_data = price_data.drop(['Unnamed: 0','New_Price','Location'], axis=1) price_data = price_data.dropna() price_data = price_data.reset_index(drop=True) price_data['Fuel_Type'].value_counts() Diesel 3195 Petrol 2714 CNG 56 LPG 10 Name: Fuel_Type, dtype: int64 price_data['Transmission'].value_counts() Manual 4266 Automatic 1709 Name: Transmission, dtype: int64 price_data['Owner_Type'].value_counts() First 4903 Second 953 Third 111 Fourth & Above 8 Name: Owner_Type, dtype: int64

# Lets split some columns to make a new feature train_df = price_data.copy() name = train_df['Name'].str.split(" ", n =2, expand = True) train_df['Company'] = name[0] train_df['Model']= name[1] train_df['Mileage'] = train_df['Mileage'].str.split(" ", n=1, expand = True).get(0) train_df['Engine'] = train_df['Engine'].str.split(" ", n=1, expand = True).get(0) train_df['Power'] = train_df['Power'].str.split(" ", n=1, expand = True).get(0) train_df = train_df.drop(['Name'], axis = 1) train_df['Mileage'] = train_df['Mileage'].astype(float) train_df['Engine'] = train_df['Engine'].astype(int) train_df.replace("null", np.nan, inplace = True) train_df = train_df.dropna() train_df = train_df.reset_index(drop=True) train_df['Power'] = train_df['Power'].astype(float) train_df['Company'].value_counts() Maruti 1175 Hyundai 1058 Honda 600 Toyota 394 Mercedes-Benz 316 Volkswagen 314 Ford 294 Mahindra 268 BMW 262 Audi 235 Tata 183 Skoda 172 Renault 145 Chevrolet 120 Nissan 89 Land 57 Jaguar 40 Mitsubishi 27 Mini 26 Fiat 23 Volvo 21 Porsche 16 Jeep 15 Datsun 13 Force 3 ISUZU 2 Bentley 1 Ambassador 1 Isuzu 1 Lamborghini 1 Name: Company, dtype: int64 train_df['Company'] = train_df['Company'].replace('ISUZU','Isuzu')

# Handling Rare Categorical Feature cat_features = [featureforfeatureintrain_df.columnsiftrain_df[feature].dtype == 'O']forfeatureincat_features: temp = train_df.groupby(feature)['Price'].count()/len(train_df) temp_df = temp[temp > 0.01].index train_df[feature] = np.where(train_df[feature].isin(temp_df), train_df[feature],'Rare')

train_df['Company'].value_counts() Maruti 1175 Hyundai 1058 Honda 600 Toyota 394 Mercedes-Benz 316 Volkswagen 314 Ford 294 Mahindra 268 BMW 262 Rare 247 Audi 235 Tata 183 Skoda 172 Renault 145 Chevrolet 120 Nissan 89 Name: Company, dtype: int64 train_df.info() RangeIndex: 5872 entries, 0 to 5871 Data columns (total 12 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 Year 5872 non-null int64 1 Kilometers_Driven 5872 non-null int64 2 Fuel_Type 5872 non-null object 3 Transmission 5872 non-null object 4 Owner_Type 5872 non-null object 5 Mileage 5872 non-null float64 6 Engine 5872 non-null int32 7 Power 5872 non-null float64 8 Seats 5872 non-null float64 9 Price 5872 non-null float64 10 Company 5872 non-null object 11 Model 5872 non-null object dtypes: float64(4), int32(1), int64(2), object(5) memory usage: 527.7+ KB train_df['Seats'] = train_df['Seats'].astype(int)

# Encoding Categorical data columns = ['Fuel_Type','Transmission','Owner_Type','Company','Model']defcategorical_ohe(multicolumns): df = train_df.copy() i = 0forfieldsinmulticolumns: print(fields) d1 = pd.get_dummies(train_df[fields]) train_df.drop([fields], axis = 1)ifi == 0: df = d1.copy()else: df = pd.concat([df, d1], axis = 1) i = i + 1 df = pd.concat([df,train_df], axis = 1)returndf final_df = categorical_ohe(columns) final_df = final_df.loc[:,~final_df.columns.duplicated()] now = datetime.datetime.now() final_df['Year'] = final_df['Year'].apply(lambda x : now.year - x) corr = final_df.corr() corr Diesel Petrol Rare ... Power Seats Price Diesel 1.000000 -0.977947 -0.113891 ... 0.292420 0.309581 0.321035 Petrol -0.977947 1.000000 -0.096114 ... -0.272662 -0.303177 -0.309363 Rare -0.113891 -0.096114 1.000000 ... -0.096618 -0.033244 -0.058408 Automatic 0.139557 -0.125612 -0.067592 ... 0.644688 -0.074554 0.585623 Manual -0.139557 0.125612 0.067592 ... -0.644688 0.074554 -0.585623 ... ... ... ... ... ... ... Mileage 0.097562 -0.130056 0.153696 ... -0.538844 -0.331576 -0.341652 Engine 0.430151 -0.410837 -0.095742 ... 0.866301 0.401116 0.658047 Power 0.292420 -0.272662 -0.096618 ... 1.000000 0.101460 0.772843 Seats 0.309581 -0.303177 -0.033244 ... 0.101460 1.000000 0.055547 Price 0.321035 -0.309363 -0.058408 ... 0.772843 0.055547 1.000000 corr[corr['Price'] > 0.4] Diesel Petrol Rare ... Power Seats Price Automatic 0.139557 -0.125612 -0.067592 ... 0.644688 -0.074554 0.585623 Engine 0.430151 -0.410837 -0.095742 ... 0.866301 0.401116 0.658047 Power 0.292420 -0.272662 -0.096618 ... 1.000000 0.101460 0.772843 Price 0.321035 -0.309363 -0.058408 ... 0.772843 0.055547 1.000000

df = final_df.drop(final_df[columns],axis=1) X = df.drop(['Price'],axis=1) y = df['Price'] X.head(5) Diesel Petrol Rare Automatic ... Mileage Engine Power Seats 0 0 0 1 0 ... 26.60 998 58.16 5 1 1 0 0 0 ... 19.67 1582 126.20 5 2 0 1 0 0 ... 18.20 1199 88.70 5 3 1 0 0 0 ... 20.77 1248 88.76 7 4 1 0 0 1 ... 15.20 1968 140.80 5

# Splitting Dataset into training and testing data X_train, X_test, y_train, y_test = train_test_split(X, y, test_size= 0.2, random_state = 0) # Feature Scaling scaler = StandardScaler() scaler.fit(X) X_train = scaler.transform(X_train) X_test = scaler.transform(X_test) best_alpha = 0.00099 regr = Lasso(alpha=best_alpha, max_iter=50000) regr.fit(X_train,y_train) y_pred = regr.predict(X_test) ac(y_test,y_pred) 0.7761524710799422

## Conclusion

In this article we have learnt about Regularization, types of penalty in regularization and techniques or types of regularization. And then we have learnt about Lasso Regression in brief and how to implement it. In our model we got the accuracy of 77% which can be further increased by hyperparameter tuning, in case you don’t have any idea about how to do hyperparameter tuning you can please refer to this previous article about how to do hyperparameter tuning.

**About the Author's:**

## Write A Public Review