Appendix A. Final questionnaire
Invitation letter (in English) ++ email/postal mail address??
Dear industrial colleague!
NTNU in Trondheim and the Simula Research Laboratory in Oslo are jointly participating in the research projects SPIKE (on process improvement) and INCO (on incremental/component-based system development). As part of this research, we wish to perform a questionnaire-based survey in Norwegian ICT industry on OTS-based system development (Off-The-Shelf, including Open Source Software and Commercial Off-The-Shelf Software). The subject is highly relevant for both research and industry.
We have already had a test run with a trial questionnaire in August 2003 with 6 persons, and 16 structured interviews in November 2003 - January 2004. We now want to expand the survey to a larger part of the Norwegian ICT sector, and to perform parallel surveys in cooperation with research colleagues at IESE in Kaiserslautern in Germany and Politecnico di Torino in Italy. The revised questionnaire has now been tested on about 15 persons in April-May 2004 in these three countries.
We now want to start the main survey, involving over 200 ICT companies (large, medium, and small) in each country. We therefore look for software developers in your company with hands-on experience with OTS-based system development. In this request, we attach an electronic version of the questionnaire as a word file. We will also contact you by phone. If you or a colleague volunteer to participate in this survey, we will send you (called a “respondent”) a userid and password to access a web-based version of the questionnaire. Although we prefer that you fill in the web-version, you can also fill in the questionnaire on paper (after printing out the word file) and return it by postal mail, or fill in the word file electronically and return to us as an email attachment.
All provided information will be treated strictly confidential. A summary of the results will be returned to the respondents as a technical report, and by a web file. Respondents will also be invited to a public sum-up seminar to discuss the findings. We will also arrange a lottery among the respondents, where the winner will receive a travel of 10 000 NOK. Two PhD fellows at NTNU, Jingyue Li and Odd Petter N. Slyngstad, will be responsible for carrying out the survey. The main responsible in Norway is prof. Reidar Conradi at NTNU.
The questionnaire is divided into 6 sections, where section 1 and 6 contain questions regarding context/background, and sections 2-5 ask for information more specific to your OTS-based project. There are ca. 40 question in total, and most are multiple-choice questions. This final version of the questionnaire will require 20-30 minutes to fill out.
So, would you be willing to participate in such a survey?
As mentioned above, we attach an electronic copy of the questionnaire. If you or a colleague volunteer to participate in the survey, we will in a few days send you a user login and password to access the web-version.
- Best regards, Reidar
Please read carefully before completing the questionnaire
Survey Objective / The purpose of this questionnaire is to survey the current state of practice in component based software engineering (CBSE) in various organizations and projects, involving the ICT companies in several countries.The background we want to measure, are hands-on experience with CBSE-development in concrete and completed projects, preferably with a maintenance phase afterwards. We want answers based on concrete experience (by your team etc.), not more general company information or a private/textbook opinion on CBSE.
Coverage / This questionnaire deals with reuse of external components, i.e. COTS or OSS components (see definitions later).
Confidentiality / The responses of this questionnaire will be handled confidentially. Whether a company has participated is also confidential.
The conclusion of this survey provides only the current state of practice in CBSE in the ICT industry. Detailed information of each company will not be included in the final report. It is not possible to draw conclusions about your company from the research results.
Language / You are expected to answer all the questions in your native language, but you may also respond in English.
Benefits / The final report of the survey will be available to all respondents, and sent by email or through a web-link. A hard copy of the report can be sent if a post address is provided. In addition, a public, sum-up seminar will be arranged.
Some general instructions / Use an “X” when you mark an answer to a question, e.g. in a table. Filling-in of the questionnaire should not take more than half an hour.
Name of respondent: Last Name (Family Name) Middle Name First Name
Age: Gender (Male/Female):
Email address: Phone: Fax:
Name of the Company /Business unit:
Postal address of Company/Business unit: / Postal Code / Town / Country
Date of response (mm/dd/yy): / Time used to fill in this questionnaire (in minutes):
NB: The above information will be treated confidentially.
Our definition of concepts used in this questionnaire:Project: A software development project, in which the respondent has been involved.
We aim for completed projects, possibly with maintenance, and possibly with several releases.
Customer: Representing either an end-user, or a main contractor to which your organization is a
subcontractor.
Component: Software components are program units of independent production, acquisition, and deployment that can be composed into a functioning system. We limit ourselves to components that have been explicitly decided to either be built from scratch, or to be acquired externally as an OTS-component. That is, to components that are not shipped with the operating system, not provided by the development environment, and not included in any pre-existing platform. That is, platform (“commodity”) software are not considered, e.g. an OS like Linux, DBMSes, various servers, or similar software. Furthermore, components usually follow some component model, as expressed by the COM, CORBA, EJB, .Net, or Web Service standards, or they can be a C++/Java library.
OTS (Off-The-Shelf) component: A component that:
· is provided (by a so-called provider) from:
- a commercial vendor (Commercial-Off-The-Shelf, COTS), or
- the Open Source community (Open Source Software, OSS).
The components may come with certain obligations, e.g. payment or licensing terms.
NB: we do not consider proprietary components developed by some other organization in
the same company, i.e. internal reuse.
· is not controllable, in terms of provided features and their evolution.
· is used as closed source, i.e. no source code is modified, even it is available.
Typical COTS components are: Vector draw professional (3D modeling), Shamman Reports for
Excel (Database reporting), Dynamic Cube (OLAP) etc.
Typical OSS components are: Xerces (XML parser), Kiwi Toolkit (Class library), OpenEJB,
Python (Interpreter) etc.
System: Product delivered by the actual project, consisting of application plus components, but again
excluding platform software.
Application: Code built in-house to provide certain functionalities of the system.
Glueware: Code to solve possible mismatch between components, or between components and
application/platform.
Addware: Code to add the functionalities which are expected but not provided by the components.
The relationship between the four last concepts is shown in the following figure:
Question for comments.Can you give an example of OTS components based on your understanding?
______
Section 1: Questions on the context of the project and system
1.1 Information about the project
The projects we want to investigate in this survey are completed software development projects using one or more OTS components. If you have experience with several such projects, please select the one that you are most familiar with, and base your answers on that project.
1.1.1 What was the total staff size of the project (full-time and part-time persons)?
Full time: ______
Part time: ______
1.1.2 How large a percentage of the project members have previous experience with
general OTS-based development __ (%)?
1.1.3 What was the starting time of the project (mm/yy) / ?
1.1.4 What was the time of first complete delivery from the project (mm/yy) ___/____?
1.1.5 What was the development environment of the project (describe textually)?
· Programming Languages: ______
· CASE (Computer Assistant Software Engineering) tools: ______
1.2 Information about the system
1.2.1 What were the main functionalities of the system (describe textually)?
______
______
1.2.2 What was the emphasis on the following characteristics of the system?
[Mark one alternative per row]
Very low / Low / Medium / High / Very high / Don’t knowa) Time-to-market
b) Effort (cost)
c) Reliability
d) Security
e) Performance
f) Maintainability
g) New functionality (first launch in the market)
h) Improved functionality (over competitors)
i) Other (please specify): ______
1.2.3 What were the major types of industry / services (application domain) of the
customers of the system?
[Mark one or more alternatives]
□ Bank/ Finance/ Insurance
□ Other private services (wholesale, retail, entertainment, …)
□ Public sector
□ ICT industry
□ Traditional industry/engineering/construction
□ Other (please specify): ______
1.2.4 What is the executing environment of the system (describe textually)?
l OS (Operating System):______
l DBMS (Database Management System): ______
l Other Supporting Software: ______
1.2.5 What is the current version/release of the system (Version/Release) ___/___?
Section 2: Questions on the motivation of using OTS components
2.1 To what extent did you expect, that the chosen OTS component(s) would contribute to achieve the following project/system targets?
[Mark one alternative per row]
Project/system targets
/ Don’t agree at all / Hardly agree / Agree some what / Agree mostly / Strongly agree / Don’t knowa) Shorter time-to-market
b) Less development effort/cost
c) Less maintenance effort/cost
d) Larger market share
e) Compliance with industrial standards
f) Keeping up with the newest technology
g) Better system reliability
h) Better system security
i) Better system performance
j) Other (please specify): ______
2.2 What were the specific motivations of using OSS Component(s) in your project?
[NB: If you did not use a OSS Component in the project,please skip this question]
[Mark one alternative per row]
Motivations
/ Don’t agree at all / Hardly agree / Agree some what / Agree mostly / Strongly agree / Don’t knowa) Reduce risk of provider going out of business
b) Reduce risk of provider changing market strategies
c) Reduce risk of selected component evolving into an unwanted direction
d) Component can be acquired for free
e) Source code is available and can easily be changed
f) It was decided by customer
g) Political reasons
h) Other (please specify): ______
2.3 What were the specific motivations of using COTS Component(s) in your project?
[NB: If you did not use a COTS Component in the project,please skip this question]
[Mark one alternative per row]
Motivations
/ Don’t agree at all / Hardly agree / Agree some what / Agree mostly / Strongly agree / Don’t knowa) Reduce risk of bad quality, because components were paid for
b) Reduce risk of poor security, because source code was closed/ unavailable
c) Provider could give adequate technical support
d) Provider may have investigated the market trends, and will give a component following them
e) It was decided by customer
f) Political reasons
g) Other (please specify): ______
Section 3: Questions on the general development processes
3.1 Please describe the main development process (e.g. waterfall, prototyping, incremental etc. – describe textually):
______
______
3.2 Who decided the main (i.e. lifecycle) process?
[Mark one or more alternatives]
□ Company/department rules
□ Project manager
□ Software architect
□ Software developer
□ Customer
□ Other (please specify): ______
3.3 When was the main process decided?
[Mark one alternative]
□ Before the “acquire vs. build” decision, i.e. before the decision whether to acquire
OTS components instead of building them in-house
□ After such an “acquire vs. build” decision
3.4 Were there any dedicated personnel in your project to keep track of the OTS component knowledge/information?
[Mark one alternative]
□ No
□ Yes
If Yes, what is the role and OTS component-relevant experience of this person?
______
Section 4: Questions on the project level risk management
4.1 What is your opinion on the following aspects of your OTS-based project?
[Mark one alternative per row]
Don’t agree at all / Hardly agree / Agree some what / Agree mostly / Strongly agree / Don’t knowa) The project was not completed according to schedule
b) Effort to select OTS components were not satisfactorily estimated
c) Effort to integrate OTS components were not satisfactorily estimated
d) OTS components negatively affected system reliability
e) OTS components negatively affected system security
f) OTS components negatively affected system performance
g) Requirements were continuously changed
h) OTS components could not be sufficiently adapted to changing requirements
i) Project could not (re)negotiate requirements with the customer, if OTS components could not satisfy all requirements
j) It was difficult to identify whether defects were inside or outside the OTS components
k) It was difficult to plan system maintenance, e.g. because OTS components had asynchronous release cycles
l) It was difficult to update the system with the last OTS component version
m) OTS components were not satisfactorily compatible with production environment when the system was deployed
n) Information on the reputation and technical support ability of provider were inadequate
o) Provider did not provide enough technical support/ training
4.2 What actions were performed during development or maintenance of the project?
[Mark one alternative per row]
Don’t agree at all / Hardly agree / Agree somewhat / Agree mostly / Strongly agree / Don’t knowa) Customer had been actively involved in the “acquire” vs. “build” decision of OTS components
b) Customer had been actively involved in OTS component selection
c) OTS components were selected mainly based on architecture and standards compliance, instead of expected functionality
d) OTS components qualities (reliability, security etc.) were seriously considered in selection
e) Effort in learning OTS component was seriously considered in effort estimation
f) Effort in black-box testing OTS component was seriously considered in effort estimation
g) Unfamiliar OTS components were integrated first
h) Did integration testing incrementally (after each OTS component was integrated)
i) Local OTS-experts followed the OTS component updates and possible consequences actively
j) Maintained a continual watch on the market and looked for possible substitute components
k) Maintained a continual watch on provider support ability and reputation.
Section 5: Questions on the OTS component level risk management