A data scientist is a person hired to analyze and interpret complicated digital records, together with the utilization statistics of a website; particularly so that it will help an enterprise in its decision-making.
An analytical model is a mathematical model that is designed to carry out a particular task or to find out the probability of a selected event i.e. the solution to the equations used to describe modifications in a system can be expressed as a mathematical analytic function.
According to Layman, an analytical model is simply a mathematical presentation of an enterprise problem. A simple equation y=a+bx may be termed as a model with a group of predefined input data and desired output.
Scalable and efficient analytical modeling is severely consequential to enable the business to use those techniques to ever-more sizably voluminous data sets for reducing the time taken to carry out these analyses. Accordingly, models are engendered that put into effect key algorithms to determine the solution to our quandary business.
Machine learning is part of computer science that enables the computer system to learn with data in order to improve the performance of the specific task. The machine learning is intently related to computational statistics, which also focuses on prediction-making thru using computers. Data scientists use different kinds of machine learning algorithms in order to analyze data patterns in big data that lead to actionable insights. Machine learning algorithms can be classified into supervised learning, unsupervised learning, and reinforcement learning.
Supervised vs unsupervised vs Reinforcement Machine Learning Model:
Most of the practical machine learning uses supervised learning. It is commonly used to discover the data pattern in big data. In supervised machine learning, we’re given a statistics set and already recognize what our accurate output should look like, having the concept that there is a clear relationship between the input and the output data
On the other hand in unsupervised machine learning, we have little or no idea what our result should look like. In unsupervised machine learning model, we do not have the idea about the model output or there are no target attributes. In unsupervised machine learning model, there is no difference between explanatory and dependent variables. The models are created to find out the intrinsic structure of statistics. Whereas reinforcement learning (RL) is an area of machine learning and artificial intelligence stimulated by behaviorist psychology involved with how software agent and machine automatically determine movements in an environment so that it will maximize a few notions of cumulative reward.
Simple reward feedback is required for the software agent or machine to analyze its behavior; this is known as the reinforcement signal. There are many different machine learning algorithms to handle these issues. We can classify these algorithms in different categories.
Supervised Machine Learning:
- Linear Regression
- Logistic Regression
- CART(Classification and Regression Trees)
- Naïve Bayes
- KNN(k-nearest neighbors algorithm)
Unsupervised Machine Learning:
- Apriori
- K-means
- PCA (Principal Component Analysis)
Here we are going to discuss about ten most important algorithms that every data scientist should know.
1. Linear Regression: –
Linear regression is one of the well known and well understood supervised machine learning algorithms. Before discussing linear regression first let’s know about regression. Regression is the statistical way to establish a relationship between a dependent variable and a set of independent variables through fitting the discovered records points on a linear equation. For example
Y= a+bX
This is the basic example of linear regression. Here we are establishing the relationship between x andy, where X is independent variable and Y is the dependent variable.
Where X= Independent variable
Y= dependant variable and
b =slope of line
a = intercept (When X=0).
2. Logistic Regression: –
Logistic regression machine learning is also borrowed from statistics like linear regression. It is the process of finding the relationship between input and output data but in this case, the output will be a binary value (0/1, yes/no, true/false).
For example, Will there be a jam in a certain area in Mumbai is a binary variable? The output is a binary variable (yes/no).
The chance of incidence of traffic jam depends on different things like weather situation, the day of week and month, time of day, number of vehicles and many others. By using logistic regression machine learning, we can find the best fitting model that explains the relationship between independent attributes and traffic jam occurrence rates and predicts the chance of jam prevalence.
Logistic regression is the process of predicting the probability of final result that can have two values (0/1). The prediction is based on various attributes those may be numerical and specific. Linear regression machine learning isn’t suitable for predicting the value of a binary variable for 2 reasons:
- Linear regression machine learning can predict the value outside the acceptable range (predicting probabilities outside the range 0 to 1).
- Since the dichotomous experiments can only have one of two possible values for each experiment, the residuals will not be normally distributed about the predicted line.
On the other hand, a logistic regression produces a logistic curve, which is limited to values between 0 and 1. Logistic regression is similar to a linear regression, but the curve is constructed using the natural logarithm of the “odds” of the target variable, rather than the probability. Moreover, the predictors do not have to be normally distributed or have equal variance in each group.
But in case of logistic regression, it produces a logistic curve, that’s confined to values between zero and 1. Logistic regression is similar to a linear regression, but the curve is constructed by the using natural logarithm of the “odds” of the target variable, as opposed to the possibility. Moreover, the predictors do not need to be generally dispensed or have the same variance in each group.
3. Hypothesis Testing: –
Hypothesis testing is performed to check if a hypothesis is true or not, using the data. Based totally on hypothetical checking out, we choose to simply accept or reject the hypothesis. Whilst an event happens, it may be a trend or happens through chance. To check whether the event is an important prevalence or just by the chance, hypothesis testing is necessary to perform.
A case study:
Let us say that average marks in the mathematics of class 9th students of D.A.V School are 85. Alternatively, if we randomly pick 30 students and calculate their average marks, their average marks come to be ninety-five. What may be concluded from this test? It’s simple. Here are the conclusions:
- These 30 students are different from D.A.V school’s 9th class students, as a result, their average marks are the higher i.e behavior of these randomly chosen 30 students sample is different from the population (all D.A.V school’s class 9th college students) or these are two distinctive population.
- There’s no difference at all. The result is due to random chance i.e. we observed the average marks of 85. it could have been higher or lower than eighty-five considering there are students having average marks less or more than 85.
How can we decide which case is accurate? There are various techniques to help you to determine this. Here are a few techniques.
1. Increase sample size
2. Test for other samples
3. Calculate the random chance probability
4. Clustering Techniques:-
Clustering is the process of analyzing and diving statistical data or population into a number of groups such that data points inside the identical group are more similar to another data point inside the identical group than the ones in the different group. In simple term, the aim is to segregate group with similar trends and assign them into clusters. It is unsupervised machine learning algorithm where outcomes are unknown to the analyst.
A case study:Suppose, you are running a business and desirous to recognize preference and purchasing behavior of your customers to scale up your business. It isn’t possible to take a look at information of each customer and devise a separate business strategy for each of them. However, what you can do is to divide all your customers into say 10 different groups primarily based on their purchasing habits and preference and use a separate strategy for customers in every one of these 10 different groups. And that is what we call clustering.
Types of Clustering Techniques: – We can broadly classify clustering algorithm in two categories.
- Hierarchical clustering
- Partitional clustering
5. Decision Trees: –
Decision trees are machine learning algorithm can be used for unsupervised or supervised in which every branch node represents a choice between a number of alternatives and each leaf node represents a decision.
Types of Decision Trees:-There are two mainly two types of decision trees.
- Classification trees.
- Regression trees.
6. Principal Component Analysis: –
The principal component analysis is a statistical procedure used to reduce large data set to the small data set that contains most of the information in the large data sets without the loss of the feature of information that is conveyed by the dataset. Principal component analysis (PCA) is a mathematical procedure that transforms a number of correlated variables into a number of uncorrelated variables called principal components.
Two commonly used variable reduction techniques are:
- Principal Component Analysis (PCA)
- Factor Analysis
7. Neural Network: –
An artificial neural network is information processing system inspired by the human nervous system used for processing complex data and finding the pattern in complex data. The key element of this model is the information processing system. it’s composed of a large variety of highly interconnected processing elements (neurons) working in unison to solve specific issues. ANNs, like human beings, learn by means of the example are configured for a specific application. An ANN is configured for a specific application, including the pattern matching or data grouping, through a learning system. Learning in biological systems includes changes to the synaptic connections that exist between the neurons.
Human and Artificial Neurons – investigating the similarities:-
A lot continues to be unknown approximately how the brain trains itself to process information and data, so theories abound. Inside the human brain, typically neuron collects signal from others thru a host of fine structures referred to as dendrites. The neuron sends out spikes of electrical pastime through an extended, thin stand known as an axon, which splits into thousands of branches. on the quit of each branch, a structure referred to as a synapse converts the activity from the axon into electric effects that inhibit or excite interest from the axon into electrical effects that inhibit or excite activity inside the related neurons. Whilst a neuron receives excitatory input that is sufficiently large in comparison with its inhibitory input, it sends a spike of electrical activity down its axon. Gaining knowledge of occurs by using converting the effectiveness of the synapses so that the influence of one neuron on some other modifications.
</centerFrom Human Neurons to Artificial Neurons: –We conduct these neural networks by first attempting to deduce the essential capabilities of neurons and their interconnections. We then commonly program a software or system to simulate those functions. However because our understanding of neurons is incomplete and our computing power is restricted, our models are always gross idealizations of actual networks of neurons.
</center8. Conjoint Analysis: –
Conjoint Analysis is the survey-based statistical technique used in the market to identify customer’s preference for various attributes. The attributes can be various features such as size, color, usability, and product. Using conjoint analysis, brand managers can identify which features would customer’s preference for certain price points. Thus it is highly useful and beneficial technique for conducting market surveys in order to design a new product or pricing strategies.
9. Ensemble Methods: –
Ensemble method is used to improve machine learning outcomes by combining various models. Ensemble methods combine various machine learning techniques into one predictive model to decrease variance, bias and improve prediction. It works on the principle that, many weak learners can come together to give a strong prediction. Random forest is the currently most accurate ensemble methods available.
10. Naive Bayes Classification:-
It is a classification technique based on Bayes’ Theorem with an assumption of independence among predictors. Bayes theorem named after Rev. Thomas Bayes. it works on the conditional probability. Conditional probability is the possibility that something will happen, for the reason that something else has already occurred. By using the conditional probability, we can calculate the probability of an event using its prior knowledge.
Below is the formula for calculating the conditional probability of any event.
Where
- P(H) is the probability of hypothesis H being true. This is known as the prior probability.
- P(E) is the probability of the evidence(regardless of the hypothesis).
- P(E|H) is the probability of the evidence given that hypothesis is true.
- P(H|E) is the probability of the hypothesis given that the evidence is there.