Research Report
Student Name
University Name ()
Course Name
Instructor Name
Date ()
Introduction
Statistical analysis is the basics of evidence-based research. Scholars and researchers employ statistical and data analysis techniques to evaluate differences, correlations, and similarities in existing problems and improve already existing research. Additionally, data analysis is employed in research studies to support the formulated hypothesis concerning a specific topic. In this research study, data analysis will be used to access respondents’ responses to job boards based on the respondent’s characteristics. Data for the research study has been obtained from the stack overflow 2019 survey. The survey contains respondents’ responses concerning various aspects related to Employment, open-source coding, working conditions. Coding as a hobby, etc. The dependent variable in this research is the use of job boards by the respondents.
Other than sharing information and solving problems related to coding and technology, stack overflow members do advertise for jobs targeting specific applicants. The aim of this report is to assess the factors as indicated by the respondents that affect the use of job boards by stack overflow members. In order to investigate this problem, statistical techniques such as the chi-square test of independence and random forest classification models will be applied. Accessing the factors that affect job boards’ use will help reduce job posting redundancy and improve applicants targeting.
The significance of a statistical model is accessed using specific parameters. The random forest classification model is accessed using the accuracy scores as well as the coefficients of the variables. An accuracy level of above 0.5 is recommended since it implies that the fitted model can be able to account for over half of the variance occurring due to model fitting and estimation. The data, therefore, has to be split randomly to account for training data and testing data.
Problem Statement
Normally, job posting in Stack Overflow needs to be posted on one board to target suitable candidates for the job. Nevertheless, this is not always the case since jobs are posted in numerous places, and this attracts different candidates, both suitable and unsuitable for the posted jobs. Factoring in the candidates’ characteristics, accessing the factors influencing the use of job boards can lead to better targeting of candidates, reduce job posting redundancy, and probably decrease the number of unfit applicants significantly.
Research Method
Several statistical research methods exist in the research concept. Therefore, identifying the correct research method to be employed in a research study is crucial to overall results and the accuracy of the research report. The nature of the research study depicts the study’s nature and the research methods to be employed. Based on the study’s objectives, that is to identify the factors affecting the respondents’ response to the use of job boards; the research study, therefore, is descriptive research. The methodology employed is the survey methodology. A survey methodology deals with exploring and describing respondents’ view on a particular topic or a question.
Qualitative research methods that deal with quantitative measurements try to answer quantitative research questions laid down (Bryman, 2016). The research problem also entails prediction through a random forest model. Thus, quantitative techniques will be used. Research methods, therefore, depend on the objectives of the study, research question, and the structure of the data.
Research Questions
The research questions formulated in a research study aids in achieving the laid down research objectives. The research questions, therefore, need to be stated specifically and precisely. Additionally, they should be measurable achievable and should be well structure and constructed to comprehensively address the research aims. Normally, they are stated as a question that requires a specific approach.
- Is there a significant relationship between respondent’s response to employment status, the response on open source, codding as a hobby, and the use of job boards?
- Is there significant evidence to show that the respondent’s response on Employment, open-source, and codding as a hobby provides enough evidence for predicting the respondents’ response on the use of job boards
- Does the data provide enough evidence for predicting the use of job boards?
Sample
In statistical research, a sample is a representation of the entire population in a research study. Generally, analysis done on a sample is used to generalize the results to the entire population. Sampling is always recommended whenever the researcher cannot be able to access every element in the population. Determining the sample size to be used in the research study is of importance since the size of the sample affects the analysis results. Increasing the sample size increases the accuracy of the results, while decreasing the sample size decreases the results’ accuracy. A sample size of 5500 respondents from three countries was used for the analysis. All the respondents participated in the Stack Overflow survey of 2019. The sample size in a research study is a very important aspect of the analysis. The analysis results and the model accuracy depend on the size of the sample. Statistically, the accuracy of the model increases with an increase in sample size.
Analysis Method and Limitations
Data analysis techniques range from descriptive analysis and inferential analysis and. Descriptive statistics describe the data to offer a better understanding of the data before complex analysis is performed. Analysis techniques for categorical data include frequency tables and bar graphs. A frequency table presents the frequencies of each class in the categorical variable. Additionally, a contingency table can also be used to analyze two categorical variables using their frequency distribution. The Chi-square test of independence investigates the presence of any statistically significant difference in two categorical variables. For instance, in order to investigate whether there is a difference in the respondent’s response to Employment and the respondent’s response to student status.
The study’s p-value was set to be the standard value, 0.05, and an alpha level of 0.5. statistically significant results are identified by the p-value score being less than the p-value. A significant difference in the categories distribution exists, and this highly affects the accuracy of the analysis. The presence of missing values also reduces the sample size of the data, which might affect the accuracy of the overall results (Hennink et al., 2020).
Descriptive Statistics
Fig 1.0: A bar graph of variable Hobbyist
Fig 1.1: A bar graph of variable Employment
Fig 1.2: A bar graph of variable Opensourcer
Fig 1.3: A box plot and a histogram of the variables Age and CodeRevHrs
Fig 1.4: A bar graph of the dependent variable Job boards
Results
Data analysis results show that s a significant association between respondent’s responses to Employment and the use of job boards. Corollary, the variable Hobbyist is also statically associated with the job boards as indicated by the chi-square test of independence. The variables opensource is not associated with job boards as the p values from the chi-square test of independence are greater than 0.05.
Table 1.0: Chisquare test of independence on Employment and use of job boards
| Pearson’s Chi-squared test |
| data: table(Mydata$SOJobs, Mydata$Employment) |
| X-squared = 193.05, df = 10, p-value < 2.2e-16 |
Table 1.1: Chisquare test of independence on code as a hobby and use of job boards
| Pearson’s Chi-squared test |
| data: table(Mydata$SOJobs, Mydata$Hobbyist) |
| X-squared = 4.6969, df = 2, p-value = 0.09552 |
Table 1.3: Chisquare test of independence on opensourcer and use of job boards
| Pearson’s Chi-squared test |
| data: table(Mydata$SOJobs, Mydata$OpenSourcer) |
| X-squared = 110.01, df = 6, p-value < 2.2e-16 |
Discussion
Table 1.4: Random forest model
| mtry Accuracy Kappa AccuracySD KappaSD |
| 1 1 0.5468891 0.00000000 0.0005534559 0.000000000 |
| 2 2 0.5543406 0.02660861 0.0037644631 0.009004333 |
| 3 3 0.5604795 0.07642199 0.0096809944 0.022069815 |
| 4 4 0.5679279 0.11320100 0.0106714702 0.026607434 |
| 5 5 0.5556618 0.09858260 0.0142558528 0.023421394 |
| 6 6 0.5569730 0.10273021 0.0086065024 0.011295122 |
| 7 7 0.5508357 0.09678513 0.0061475960 0.022134218 |
| 8 8 0.5552224 0.10733549 0.0122102559 0.016416118 |
| 9 9 0.5552186 0.10920201 0.0102294750 0.013260776 |
| 10 10 0.5530256 0.10697816 0.0083220774 0.022922332 |
Fig 1.5: Model accuracy
The chi-square test of independence performed on the explanatory variables showed a statistically significant association. A random forest classification model was fitted to identify whether the association is statistically enough to predict the student status. Data was divided into two parts, the training and the validation data in the ratio of 70% to 30%.
When the model was applied for prediction, an accuracy level of 54% was achieved. Further tuning the parameters, the best model accuracy was 54%. Statistically, the model was significant as the accuracy level was above the 50% level. This indicates that the respondent’s response to Employment, opensource, opensourcer, and Hobbyist provides enough evidence to predict the response to job boards’ use.
Prediction results
| Accuracy : 0.5449 |
| 95% CI : (0.4996, 0.5896) |
| No Information Rate : 0.5265 |
| P-Value [Acc > NIR] : 0.221 |
| Kappa : 0.0773 |
| Mcnemar’s Test P-Value : <2e-16 |
Recommendations for Future Research
The analysis presented in this research reports conveys a better understanding of the use of job boards with regards to respondent’s characteristics. For future researchers and authors, performing a random forest on various variables, for instance, more than four variables, would be recommended to identify the significant variables in predicting student’s status. Additionally, other variables like education, career field, and satisfaction would set some insights in the respondent’s responses.
Secondly, conducting the analysis of various countries to identify whether there exist any variations in the results as per the country of residence. Other than the random forest model, other researchers and scholars would also be recommended to fit other statistical models such as a logistic regression model, neural network, etc.
Conclusion
The respondents’ responses on the use of job boards are associated with respondents response on Employment, Hobbyist, Opensource, and Opensourcer. The association is significant enough to predict how respondents would respond to the student’s status. The respective p values that are all less than 0.05 support this. A random forest classification model effectively models the association of the use of job boards and their respective explanatory variables. Most of the respondents were employed full time representing 3907 respondents. Interestingly, the respondents who coded for hobby were significantly more than those who coded out of a hobby. Those who coded as hobby represented 4507respondents.
The random forest accuracy was 0.54, implying that the model could account for about 54% of the variance occurring to job boards’ use as a results of model estimation. The model indicates that’s the explanatory variables chosen are statistically significant in predicting job boards’ use.
References
Rodriguez-Galiano, V. F., Ghimire, B., Rogan, J., Chica-Olmo, M., & Rigol-Sanchez, J. P. (2012). An assessment of the effectiveness of a random forest classifier for land-cover classification. ISPRS Journal of Photogrammetry and Remote Sensing, 67, 93-104.
Stack overflow annual development survey (2019). https://insights.stackoverflow.com/survey/
Bryman, A. (2016). Social research methods. Oxford university press.
Hennink, M., Hutter, I., & Bailey, A. (2020). Qualitative research methods. SAGE Publications Limited.