Classification of instagram fake users using supervised machine learning algorithms

Received Oct 23, 2019 Revised Nov 26, 2019 Accepted Dec 9, 2019 On Instagram, the number of followers is a common success indicator. Hence, followers selling services become a huge part of the market. Influencers become bombarded with fake followers and this causes a business owner to pay more than they should for a brand endorsement. Identifying fake followers becomes important to determine the authenticity of an influencer. This research aims to identify fake users' behavior and proposes supervised machine learning models to classify authentic and fake users. The dataset contains fake users bought from various sources, and authentic users. There are 17 features used, based on these sources: 6 metadata, 3 media info, 2 engagement, 2 media tags, 4 media similarity Five machine learning algorithms will be tested. Three different approaches of classification are proposed, i.e. classification to 2-classes and 4-classes, and classification with metadata. Random forest algorithm produces the highest accuracy for the 2-classes (authentic, fake) and 4-classes (authentic, active fake user, inactive fake user, spammer) classification, with accuracy up to 91.76%. The result also shows that the five metadata variables, i.e. number of posts, followers, biography length, following, and link availability are the biggest predictors for the users class. Additionally, descriptive statistics results reveal noticeable differences between fake and authentic users.


LITERATURE REVIEW
As stated earlier, fake users can also contain human users [6], and this expands the definition of a fake user. However, many studies consider fake users to be bot users only. The commonly used fake project dataset [18] was acquired from bot users market and CAPTCHA validation, so it contains only bot users. A list of fake users' classification is presented in Table 1. Most studies were using supervised, features-based detection method, and used Twitter as the platform. Supervised Machine Learning (ML) techniques to identify fake accounts in the fake project dataset were done in [15,16,19], with different features set. Unlike Twitter, Facebook is richer in terms of media-related features. Features such as tags, shares, comments, and likes (given and received) can be used for identification [20]. There is one report [12] that used Instagram for fake accounts classification. However, only metadata features were used, and the fake or authentic user decisions are based on human judgement, instead of getting the fake users from bot-selling markets. Instagram doesn't seem popular for research, despite its rapidly increasing popularity in influencer marketing [3]. Abbreviations: ML (machine learning), SVM (support vector machine), RF (random forest classifier) Some features are derivatives of others, such as "has sent < 50 tweets" and "has never tweeted" derived from tweets count [10]. Below is the list of features code used in the Similarities are important differentiators of fake and authentic users, such as friends [21], activities [23], names similarity [17], posts similarity [10]. Friends similarity can be used because fake accounts tend to have similar friends to each other [21]. However, it is not practical if the fake users are coming from different countries and have different set of friends. Activity similarity [23], on the other hand, is more suitable for detecting malicious actions. Twitter data pattern analysis [22] revealed that bot accounts were almost always created in batches within a very short interval. They also have similarities in screen names and post update frequencies. This kind of pattern similarity is also used by [17] to identify fake accounts, using clustering and supervised ML.

RESEARCH METHODOLOGY
This research starts with fake users and authentic users data collection. All private users were removed, because only user's metadata can be acquired from them, not media data. Available metadata on Instagram are username, full name, biography, link, profile picture, number of posts, following, followers. After data collection, these features will be extracted, and the correlation analysis will be carried out. After setting up the features to be used, machine learning algorithms will be used to classify the users. The complete research methodology is shown in Figure 1. There are four classes of users to be identified in this research, i.e. real users, and three classes of fake users (active, inactive, spammer). These classes are based on manual observation of their behaviors.

DATASET
This section will discuss about data collection and annotation, features extraction method, correlation analysis between features, and user behavior analysis based on average of each feature of each user class.

Data collection
The dataset was collected using web scraping from third-party Instagram websites, to capture their metadata and up to 12 latest media posts from each user. The collection process was executed from September 1 st , 2019, until September 20 th , 2019. The dataset contains authentic users and fake users, which were filtered using human annotators. The authentic users were taken from followers of 24 private university pages (8 Indonesian,8 Malaysian,8 Australian) on Instagram. To reduce the number of users, they are picked using proportional random sampling based on their source university. All private users were removed, which is a total of 31,335 out of 63,795 users (49.11%). The final number of public users used in this research was 32,460 users.
There are reasons for choosing private universities, i.e. (1) private universities tend to have lesser followers if compared to public universities, so the followers are most likely more authentic, (2) universities have fewer tendencies in buying followers, compared to influencers, (3) engagement rate of private universities are higher than public ones. Engagement rate can be used to measure the authenticity of followers [8,24]. To prove the reason (1) and (3), the top 5 public and private universities in Malaysia are sampled and measured with FakeCheck.co. The average engagement rate (likes divide by followers) of private universities is 2.99%, compared to 2.43% of public universities. The average follower of private universities is 16,052, compared to 18,799 of public universities.
The fake users were collected by buying followers from Indonesia sellers, from various sources (from Instagram and Kaskus forum). In early observation, these followers satisfy the three types of fake users mentioned in section 1. Sometimes, bot users are quite obvious, usually they come with random names with random numbers in the username [17], no profile picture, no post, no biography. The data on the followers used in this research are from across the world. A dummy Instagram business account for the followers to follow was created to carry out this research. According to the analytics data in the dummy account, the top 5 countries of the fake followers are Indonesia (17%), India (13%), Turkey (9%), Pakistan (8%), Russia (7%). This indicates that this follower selling service is worldwide, and the results of this research are less likely biased towards specific countries.
There were two experiments to be conducted, i.e. with 2-classes classification (fake or authentic user), and with 4-classes. Thus, to balance the dataset, some authentic users are removed in the 4-classes classification. The removal is done by using K-Means to 10 clusters and proportional random sampling. From each cluster, 1,120 users will be picked randomly, as detailed in Table 2. This sampling is intended to preserve different user behavior. The 2-classes classification doesn't require annotation since the authentic users and fake users are coming from different sources. The 4-classes classification requires human annotation for the fake users' class. A simple web page was built to provide UI for the human annotator to annotate. The details of dataset numbers are provided in Table 3.

Dataset annotation
For the authentic users, to verify the authenticity, three human annotators were assigned to flag suspicious (or fake) users. Based on the majority voting, if two or more annotators flagged a user, the user will be removed. However, this relies on the human judgement. One more accurate approach is the CAPTCHA validation [18], which takes a lot of time to collect a large number of users. To classify the fake users to three classes, the same three human annotators were assigned. Prior to the annotation, some characteristics of each class of fake users were defined, i.e. (1) Active users, characterized by a high number of followers, have a lot of posts but usually short or no caption, or smiley-only caption. (2) Inactive users, which usually have no profile picture, and/or biography, and a low number of posts. (3) Spammers, which usually post a lot of images within a short interval. These characteristics serve only as hints, not mandatory conditions, for the annotators. The final decision is made based on majority of the voting. If there is no majority vote (6.31% occurrence), the annotator #1 decision is used as the priority.
A previous study [25] showed that there are 85% of users (from various Instagram users) who post up to 30 images. In our dataset, there are 56.4% of fake users who post up to 30 images, and 30.4% who post up to 5 images. For the spammers, upon early observations, there are generally three types of their posts, i.e. promotional or giveaway images, posts that attract new followers ("follow to follow" kind of), or any post that has high similarity with his/her other posts. Spammers can also be identified by post interval. In our media dataset, there are 22.15% of fake users who has post interval as short as 10 minutes between their own two latest posts. Overall, these facts are some additional hints for differentiating fake users to different classes.

Features extraction
The list of user features used for the classifier model is explained in Table 4. There are five different sources of the features, i.e. metadata, media info, engagement, media tag, media similarity.

Correlation analysis
Correlation is a measure of association between variables, and high correlations among the independent variables produce an unusable model [26]. The bivariate correlation method will be used to produce the Pearson correlation values. The results are shown in Table 5. As shown in Table 5, there is no strong correlation (0.7 and up) between the variables. The highest values of correlation are bl/lin (0.47) and erc/erl (0.44), which are considered as moderate (0.40 to 0.69) and acceptable [27]. The correlation between lin (availability of link) and bl (biography length) shows that users with long biography information will most likely put a link. On Instagram, users can only put one link in the biography, and it's usually used to introduce his/her website or other social media. The erc (ER-comments) and erl (ER-likes) correlation shows that the number of likes correlates linearly with the number of comments. The three other variables with highest correlation, which is weak correlation (0.10 to 0.39), are cl/cz (-0.35), cl/bl (0.35), fo/hc (0.34). The cl (caption length) and cz (caption zero) are predictable, but the cz is used to strengthen cl. Fake users will usually post media without almost zero caption (<=3 characters, either no caption or smiley only caption). In the dataset, the percentage of media posted by fake users with almost zero caption is 32.8% (123,569 of 376,357), whereas for authentic users it's only 13.8% (63,470 of 460,923). The correlation between cl (caption length) and bl (biography length) indicates that users with long captions in media will probably put longer biography as well. The correlation between fo (follower keywords in hashtag) and hc (hashtag count) is predictable since fo is a subset of hc.

User behavior analysis
In this section, analysis of the different behaviors between authentic and fake users, and between the three classes of fake users is detailed. Statistics analysis results are shown in Table 6 and Table 7. In contrast, fake users have a lower followers count, but a higher following count if compared to the authentic users. This indicates that fake users like to follow others to increase their presence. -Engagement (erl, erc): Fake users will receive more likes if compared to authentic users. However, in terms of comments, authentic users receive more. This indicates that receiving comments is harder, so the authentic users win in that case. In terms of likes, fake users generally have other fake users following them [21], so they can receive automated likes.

CLASSIFICATION
In this step, five supervised machine learning algorithms are used for the classification tasks, with the 17 mentioned features. The classification will be divided into 2-classes and 4-classes classification. The outcomes of each classification are the standard performance measures [28], i.e. accuracy, precision, recall, F-measure, ROC curve. Commonly, a smaller number of classes will lead to better accuracy, as shown in comparison in [29]. The result of classification algorithms with 10-fold cross-validation is shown in Table 8. As shown in Table 8, Random Forest consistently outperforms other algorithms. Interestingly, while other algorithms struggle in the 4-classes classification, Random Forest can perform even better than the 2-classes counterpart. Also, the features importance can be acquired from the Random Forest calculation, as shown in Table 9. The features importance result is important to understand what the highest predictors are to differentiate fake users from authentic users. An additional insight can be acquired by combining the result of Table 9 and Table 7. For the 2-classes classification, the values of flg and lin are among the top five important features, as well as the top five behavioral differentiators between authentic and fake users. Table 9. In the 4-classes classification, pos, flw, bl, flg are the four highest predictors. In the 2-classes classification, pos, flw, lin, flg, bl are the five highest predictors. These are all metadata values which are easy to acquire, even for private users. In the next experiment, classification using metadata-only variables will be carried out, to see if it is possible to classify fake and authentic users using only metadata. Table 10 shows the result of 2-class and 4-class classifications using six metadata variables (pos, flg, flr, bl, pic, lin). The classification with metadata as shown in Table 10 produced a less accurate result if compared to the classification with all features included as shown in Table 8. The highest accuracy that can be achieved in Table 10 was 79.66% (for 2-classes) and 59.14% (for 4-classes). The lower accuracy result of the 4-classes classification is because the differentiation of fake user types highly relies on media data.

DISCUSSION
The dataset analysis result provides some important insights, such as users with long biography will more likely put a link as well, and the number of likes is linear with the number of comments. The biography and link itself, are two of the five highest differentiators of authentic and fake users, with the importance of 0.35 and 0.37 (for 2-classes classification), respectively. Fake users have less biography length (average 41.3 characters) if compared to authentic users (average 73.9 characters), and less likely to put a link (12% vs. 45%). Another important differentiator is, fake users have a higher number of following if compared to authentic users, but in contrast, fake users have a lower number of followers if compared to authentic users. The provided 17 features for classification are proved to be sufficient to differentiate authentic and fake users. The highest classification accuracies for 2-classes and 4-classes are 90.09% and 91.76%, respectively, both using Random Forest. Furthermore, the features importance result indicates that the most important predictors in the classification are all metadata features, i.e. number of posts, number of followers, link availability, number of following, and biography length.
The high importance of metadata features leads to another experiment, which is classification using metadata-only features. This classification produced the highest accuracy of 79.66%. The advantage of the classification using metadata-only is, the metadata are easily acquirable, even for private users. This is a huge advantage since in the raw dataset, 49.11% of the users are private users. Thus, even though using the full features can produce a better classification accuracy, the classification with metadata-only is still acceptable given the fact that a lot of Instagram users are private users. Furthermore, collecting metadata from many users is much faster than collecting both metadata and media data.

CONCLUSION
The outcome of this research is important for business owners who look forward to finding influencers for brand endorsement. Unlike Twitter, which is used by most research, Instagram has richer features in media sharing, and it is proven to be the most used platform for brand marketing. Possible improvement for this research is the inclusion of text analysis in both caption and comments, relation graph analysis, and image analysis. Some captions of the fake users are irrelevant, such as "follow me", and some comments given to them are also coming from fake users. By doing this, the fake users can avoid being banned by Instagram by keeping good engagement levels. Image analysis is also useful since many spammers post text-based images. Graph analysis is also helpful for the classification since some of the fake users are in the same circle of relation.