todogroup / osposurvey Goto Github PK

• no more than 5-10 questions
• goal is for companies to understand their reputation in open source

Q14 On a scale of 1-5, how business-critical is your open source program to the success of your engineering or product teams?

Somewhere in Part 2: Does your company monitor cross-organizational open source dependencies?

• Yes - informally
• Yes- formally
• I don’t know
• No

This could also be turned into another line item in Part 1/Question 3, although I think it would be interesting to ask this separately as a more operationally focused question

What is your company budget to sponsor open source organizations

The list of companies in Question 32 inserts bias into the survey that is perpetuated when discussing the survey results. This question should be dropped in favor of a more open-ended question that doesn't bias the results.

I'm opening this issue in response to how I saw the data from this survey presented by The New Stack at All Things Open 2020. I'll post a link to the recorded talk once it has been posted online, and add additional comments when I can offer screenshots and precise observations.

We are making final modifications to the OSPO survey and would like to get the SC to sign off :)

Hi, coming here from reading https://thenewstack.io/research-shows-open-source-program-offices-improve-software-practices/.
Obviously the survey sampled only a tiny fraction of all companies in the world.
Please say something about how the survey was distributed / what kinds of companies was it targetted at?

Even if a reader trusts your conclusions "the findings are not the result of self-selection bias", i.e. that it's a representative sample, it's unclear a sample of what.
For example there are probably no mom-and-pop-pizza shops here, or other completely offline businesses here... Which I agree is out-of-scope to ask about OSPO and such 😉, just making the point there some implicit scope and knowing it would be useful to the reader.

A bar chart of verticals [and company sizes] could go a long way towards answering this.
chart3 has number of answers by vertical:

But is that number of individual respondents, or number of companies?

BTW, are all the graphs weighted by number of respondents from same company / company size / normalized to 1 per company?
"One hundred and thirty responses were excluded because they appeared to come from the same company" — so I guess 1 per company, all weighted equally?

Hey @LawrenceHecht, can you investigate as we have some data missing apparently.

Part 1, after Q4 suggest adding the following question. The categorization here follows a framework from Ciesielska and Westenholz’s “Dilemmas within commercial involvement in open source software”:

For your company’s core open source investment, would you describe your level of engagement as:

• Imitating: Imitating and translating ideas from open source communities (duplicating incentives, knowledge-sharing within the firm, user-involvement)
• Using: Acting as a "community customer" using the open source software and sometimes also supporting the community with money
• Combining: Combining proprietary software with open source software
• Creating: Leading open source projects (managing, creating code, supporting)
• Contributing:Participating in open source projects led by a community (creating code, supporting the project)
• Co-Leading:Members of of open source communities (co-managing the community, creating code, supporting the community)

maybe a question around the employment agreements structure for open source contribution

If we think "use" beyond what might be in the releasable artifact to be the infra that is running that work this is not a new trend. What I think is interesting is we would pull in works to do some of this plumbing with license and security implications, but we are moving to a place where it is an opaque endpoint under commercial terms. How can we make sure that even if we use a managed K8s service how can we make sure the survey can reflect not just us using React, but also the CSP capabilities.

This has possible implications to declared/observed licensing for a work, attributions, and understanding your end to end supply chain.
I also think the ability to influence a project gets weird since your lens is a commercial distribution.

How can an OSPO include a commercial distribution on the end of an IP endpoint as part of our remit as it is still open projects and standards at heart.

Q42. Of the following options, what are the top three benefits your company receives by using open source software?

Presenting the results of this question are confusing. If kept it, can we make sure the choices mirror what is being asked in 1-2 other big studies?

Q38. What is the preferred license for your company's open source projects?

Data wasn't that interesting, so it is a candidate for removal. If kept, we need to make this question required. Also, we need to clarify whether "no license" means "no preference" or a preference not have a license.

Last year we said the companies were a sub-set of TODO Group members. Are we good with using that justification again? If so, then we can't include Oracle in the question.

Hi folks, we were reading this in our organisation for few things around open source orgs, thanks for the great work done. Can you please tell me how can I find the regions studied or organisations locations, did you store the locations in the questionnaire?

• Do you know what an OSPO is, and/or how familiar are you with the term? This would include a definition. Asked at TODO Group meeting.
  ** LH's note: We originally didn't include a definition so we didn't bias the question about what a program's responsibilities are. 2) We know that among those that don't have plans for an open source program, only 21% said that it is because they were unfamiliar with the term.
• Do you leverage open source for marketing/thought leadership purposes? (suggested by @gravax)
• Do you integrate open source participation into your societal responsibility activities? (suggested by @gravax)

Hello, just reviewed this pie chart (chart 11), shown below.
The pie chart for the sponsors of open source foundations is wrong, for the 'rarely or never' contributing percentage. I intuitively discovered this because the percentages cannot add up with max rounding (23.5%+27.5%+49.5% is over a 100).

Rarely or never contributing: 43/191 = 22.51 -> 23%

I have crosschecked the counts to be correct in the csv (never/rarely=43, sometimes=53, frequent=95)

Following up from twitter conversations:

https://twitter.com/todogroup/status/1035316044573880320

Feedback from the TODO Group meeting 2019-09:

Ask people whether they know/are familiar with what an OSPO is (and include a definition of it).

Do we mean this to include projects a company maintainers under their copyright? downstream versions a company maintains? upstream projects in a foundation or other organisation where the company employees act as maintainers? Or are we okay with it being intentionally ambiguous?

Where is the open source program or initiative located within the organization? If the effort is informal, answer based on or who the primary organizers report to.
--40% said Software Engineering and Development. 39% said IT or Office of the CTO. This variable wasn't strongly predictive of anything else.

What are the ways your open source program quantifies success? (Check all that apply)
--Make this about the top three metrics?
--57% said "open source culture within the company", which I still don't think is a real metric.

What are the top three challenges your open source program faces? (Choose three)
--We didn't create a chart using this data, so it couldn't have been that useful.

Has the open source program had a specific impact on your company's DevOps practices or software architecture?
--63% said yes.
--Either get rid of it or change so it asks about a "positive impact".

Where will the open source program or initiative be located within the organization? If the effort is informal, answer based on who the primary organizers will report to.
--Included in the list above, but also don't ask it of the people who have plans to create a program.

What are the top three ways your open source program will quantify success? (Choose three)
--Make this about the top three metrics?

How often does your average application development team release code into production?
--This data wasn't that useful -- there was little correlation with other data.

What is the average time between major product releases?
--This data wasn't that useful -- there was little correlation with other data.

What kinds of tools does your company use to manage open source code repositories? (Check all that apply)
--While interesting, not as timely as before.

1. Are you in the business of developing or selling software applications?

• Yes
• No

2. If you use open source software, how do you use consume open source software?

• Do-it-Yourself (DIY)
• DIY with support from an open source software vendor
• Through an open source software vendor, with support
• Through an open source software vendor, without support
• Through a managed service provider/ or a consultant

3. If you are using open source and you are not selling software applications, why do you use open source software?

• To save on licensing costs
• To provide flexibility and choice on software vendors
• Because my managed service provider recommended so
• As a negotiation tactic while dealing with software vendors
• Other

4. If you are NOT using open source, why are you NOT using open source software?

• Lack of stability in the software
• Lack of required features
• Lack of support
• Lack of expertise
• Unpredictability in TCO estimates
• Other

Q39. Which of the following open source compliance methodologies and initiatives does your organization utilize or participate in?

ClearlyDefined
FOSSology
OpenChain
Software Package Data Exchange (SPDX)
Other (please specify)

If included in 2020, make this question required. Change question so that it asks whether or not they have any methodology or initiatives besides "homegrown", manual, etc.

A TODO item for Chris

• number of components
• % in products

Yes, + listing of specific tools or areas (compliance, security scanning etc)

Importing the CSV file to a spreadsheet, you see that some columns (K, AM for instance) contain data interpreted as dates that should be numbers. Looking at the CSV source, you see the same thing there. It looks like someone imported data into their spreadsheet software then exported it as a CSV without verifying their software hadn't "helpfully" reformatted the data for them. For example, note the "Feb-50" in the CSV entry below:

11696528278,6/14/2020 16:31,6/14/2020 16:41,Consuming open source code in products or services,,,,Collaborating with peers across open source projects and/or foundations,,,Feb-50,Rarely,Frequently,Never,Never,Never,Never,Sometimes,1-20%,No,No,No,No,No,No,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Organization�s open source use and participation is too small to need one,,,,,,,No,,,,,,,,,,,,,,,,,,,,,,,Organization's open source use and participation is too small to need one,,,,,,,,Very Poor,Don't know,Don't know,Above Average,Above Average,Below Average,Excellent,Very Poor,Don't know,Don't know,Don't know,Very influential,No,Neutral,No,,0,Don�t know,5-Jan,No,No preference,,,,,,,,,,,,,,,,,,,,,,,We do not utilize a tool or methodology for open source compliance,,,No,,,,,Homegrown solution,,,,,Functionality,Performance and stability,,,,,Total cost of ownership (licenses and developer time),,,Other IT,

Could we please get a CSV file with unmunged data for all of the columns?

/attn @LawrenceHecht

Choose from the following:

Request and/or incident management tools:

• Polarion
• Team Foundation Server
• JIRA
• Github/Gitlab/other Git service
• Other(s):

IDEs (Integrated Development Environments):

• Eclipse
• Visual Studio
• Qt Creator
• Netbeans IDE
• JetBrains CLion
• JetBrains GoLand
• JetBrains IntelliJ IDEA
• JetBrains PhpStorm
• JetBrains PyCharm
• JetBrains Rider
• JetBrains RubyMine
• JetBrains WebStorm
• Android Studio
• Other(s):

Source code management (SCM) tools:

• SVN
• Git
• Other(s):

Source code management (SCM) services:

• GitLab
• GitHub
• Bitbucket
• Sourceforge
• Other(s):

Continuous integration (CI) and continuous deployment (CD) tools:

• Jenkins
• Team Foundation Server
• Bamboo
• TeamCity
• CircleCI
• Azure DevOps
• Travis CI
• GitLab CI
• Concourse
• AWS CodeBuild
• Codeship
• Drone.io
• wercker
• Go.CD
• Semaphore
• Appveyor
• Buildkite
• Puppet
• Ansible
• Octopus
• Other(s):

Build tools, frameworks and dependency management:

• Cmake
• Yocto / OpenEmbedded
• BitBake
• Visual Studio
• Apache Maven
• Gradle
• npm
• yarn
• pip / pipenv
• Conda
• Composer
• sbt
• Make
• Apache Ant
• Webpack
• Other(s):

Package indexes and repositories

• Go Search
• npm registry
• Packagist (the PHP package repository)
• Maven repositories
• RubyGems.org
• NuGet
• Bower
• CPAN
• Cargo (crates.io)
• PEAR (PHP extension and application repository)
• PlatformIO registry
• Nexus Repository
• JFrog Artifactory
• CocoaPods
• Other(s):

Document management:

• Flowdock
• Confluence
• Polarion
• Other(s):

Testing frameworks:

• Robot Framework
• Cypress
• RedwoodHQ
• Selenium
• Serenity
• Citrus Framework
• TestRail
• qTest
• JUnit
• AndroidTest
• Roboelectric
• mochito
• Other(s):

Container technologies:

• Docker
• Cloud Foundry
• Atomic
• OpenShift
• Kubernetes
• Other(s):

There are a few instances where I'd include this, such as Part 2/Q15, Part 3/Q24, Q25, Part 4B/Q30 (hopefully I didn't miss any there). fwiw from personal experience, this was always key in my projects, and it also relates to better understanding how companies are handling cross-org dependencies

Part 2/Question 15 - add line item about better dependency and vulnerability management.

Part 1, after Q4 suggest adding how relevant is your main open source investment to your organization’s core business?

• Untapped: could substitute proprietary software without recognizable change
• Beneficial: beneficial to core business, but could deliver same value without open source but at a higher cost
• Dependent: core product couldn’t exist without it
• Promotional: open source is essential to promotion of core product (e.g. accompanying SDK or components)
• Accelerant: working with external open source community shortens time-to-value creation/capture
• Direct: main business value is derived directly off open source, e.g. GitLab

(I’m unsure how to resolve this with Part 8 question, which are more focused on operational value. This question is aimed a bit differently)

I'd like to know the staffing count for different organizations based off industry and # of employees # of engineers. Hopefully this is useful for others too!

Should a GitHub product be included in the choice set?

Q1 and Q3 answers are almost identify. I recommend getting rid of the question and taking the following choices and putting them into the Q3:

Influencing open source projects via leadership or maintainer roles
Collaborating with peers across open source projects and/or foundations

• hard dep
• core infra

Part 1, after question 4: Suggest adding: What do you consider to be your company's dominant open source investment?

Q.40 Which of the following software scanning and software composition analysis tools does your organization use?

There's no equivalent for non-application development organisations, or organisations with mixed deliverables, e.g. some applications/services and some embedded devices with less frequent update cycles. The question is also worded in a way that focusses on web applications/services, i.e. "into production", implying multiple environments that don't exist in all delivery mechanisms.

Add COVID related question, like:

"If your enterprise reevaluates budgets in light of macroeconomic conditions, how will funding of open source initiatives be viewed ?"

I like the hiring and training angle given that it was a big takeaway from last year's survey: that open source program offices are focused on developer recruitment

The hypothesis is that more business teams are "leaning in" to tech organizations' use of open source because they are open to change/see the need for it.

Do companies see open source as a way to move fast, but at the same time don't necessarily understand that you can move even faster when you're involved in the communities/ participate upstream