This is a short guide to our methodology, showing you why industry leaders trust our data.
Leading Tech companies trust us — and you should too — not only because we have been researching competitive markets for 20+ years, but because we have been producing accurate insights based on our findings and data.
See how we do it
Let us walk you through our methodology: how we design our research and surveys, and how we collect, cleanse and analyse the data.
Ask the right questions to the right audience. The right way.
First things first: we focus on asking the right questions. At SlashData, we may have data in our DNA, but we know that plunging headfirst into data is not where your quest for answers should begin.
Instead, we want to support you in your thought process and to ensure that you are asking the right questions for your business problem at hand. Knowing exactly which questions you need answered will help us specify not only what data you need, but also who we should target, and how big your sample should be.
We then translate your business questions into research questions and a survey. Your quest for answers is turning into a research plan.
Finding the Source
Most of our research at SlashData is tailored to the needs of our customers: we design an outreach plan and target the specific audience required in their research.
We partner with multiple panel providers for our custom projects, thus eliminating bias by design and we can support outreach to multiple countries, including difficult-to-reach countries, e.g. China. Our surveys are offered in multiple languages. We manage the translations.
For projects focused on software developer audiences, we can use our own panel as well as the Developer Nation community, which encapsulates many developers who use SlashData resources: forum users, newsletter recipients, podcast listeners, etc.
Our Developer Nation surveys run twice per year, bring a minimum sample of 12,500, and are helpful as they provide rich profiling data (100+ questions) on software developers as well as larger samples. We also offer the option to add custom questions to these surveys, to address specific client needs.
The surveyed developers are not the same in each wave; instead, we reach out to the global developer population afresh each time.
This way, we ensure that we accurately represent the mosaic of all different communities, backgrounds, goals and software projects out there. Depending on project needs, we can also provide a premium sample, which means that each respondent is verified through GitHub and LinkedIn.
Sophisticated data capture
Our proprietary survey tool is unsurpassed in the research space, generating more data than any other survey tool. Its “question-by-question” view means that we capture exactly the timings associated with each question, rather than the survey as a whole, while capturing key information like click position.
This respondent data allows SlashData to create a best-in-industry data cleansing methodology.
Cleansing the data
Our cleansing methodology is both extensive and thorough to ensure only high-quality data.
We include sophisticated implementations of speeding, straightlining as well as other industry standard checks, but we far surpass this with our many advanced features.
Our implementations lead the industry standard. For instance, when considering the speed at which questions are answered, we do so on a grouped level: taking into account the language (because certain languages are faster to read than others) as well as other respondent characteristics like experience.
As part of our suite of more advanced checks, we have advanced bot detection, including VPN detection, repeated VPN cycling or other unusual network features. Such advanced techniques protect your data against bots and/or click-farm activities.
Around most questions we build extensive consistency checks.
An example of this would be if you claim to use React (a front-end JavaScript based framework), but also say that you cannot code nor know any JavaScript. By checking for these logical inconsistencies we are able to detect boredom, carelessness or deception and adjust accordingly in each case.
Weighting the data
No matter how hard we try, we’re always bound to get some sample bias. Beware of anyone who says they don’t.
In certain projects, we deem it appropriate to weight the data. This is done by creating respondent weights using the ranking technique, where we weigh data to adjust for and minimise sample bias that occurred during the fielding/outreach phase of a project.
This may occur, for instance, if a particular region (e.g. South Asia) brought a disproportionate proportion of the sample. In such a case, we will reduce the weight of South Asian respondents. In practice, though, we consider many attributes (not just region) and weight in a multivariate way.
Advanced statistical reporting
We have advanced statistical reporting available.
This includes:
- Hypothesis testing - a very useful feature to see if there is a real vs only random fluctuation between two groups
- Confidence intervals - which would give high and low estimates of usage.
Data you can trust
From collection to analysis and presentation, we ensure that our data and insights provide the most possibly accurate representation of the audience we are surveying and the market as a whole.
This way, you may make decisions and build a strategy that drives innovation in Tech and in your product. With confidence.
Frequently asked questions
How does SlashData decide what to ask?
We start from the business question you need answered. We translate your business questions into research questions and a survey, and that also tells us who to target and how big the sample should be.
Where do the survey respondents come from?
We partner with multiple panel providers, which eliminates bias by design and supports outreach to many countries, including difficult to reach ones such as China. For software developer audiences we can also use our own panel and the Developer Nation community.
How large are the Developer Nation surveys?
They run twice per year and bring a minimum sample of 12,500. They carry rich profiling data, over 100 questions, and you can add custom questions of your own.
How do you keep poor quality data out?
We check for speeding and straightlining, and we compare each response against the minimum read time for that question. We also run advanced bot and VPN detection, and consistency checks around most questions.