Results of Questionnaire on ITRON-specification OS Use Trends

1. Outline of the Survey

A questionnaire was used to survey ITRON-specification OS use trends, gauge the views of users, and get feedback on the kinds of activities that should be carried out in the ITRON subproject. The results were then analyzed, and are summarized below. An outline of the survey is given first.

1.1 Survey period

The survey was conducted from November 1996 through January 1997.

1.2 Survey methods and number of responses

(1) How the survey was conducted

Questionnaires were distributed by the following means, then were collected and the answers were tabulated.

• Sent by direct mail
• Distributed at an embedded systems conference
• Made available on the Internet using a Web-based form

The survey forms were distributed mainly in Japan, and these results were tabulated and analyzed. Future surveys are to be carried out overseas, as well; so a small number of questionnaires were distributed abroad on a trial basis. The data received from this trial was too small to be counted in the final results, however.

(2) Number of responses

Of the questionnaires distributed by the above means, 287 responses were received. The number of questionnaires and responses of each kind are as follows.

	Distributed	Responses	Response rate
Direct mail	803	201	25.0%
Trade fair	996	64	6.4%
Internet	---	22	---

The response rate from direct mailings was lower than last year's survey (at 35.5%), which relied on direct mail alone. The likely reason is that this year's survey for the first time was limited to one TRON subproject, ITRON, and was directed at engineers working in the embedded systems field; so anyone else receiving the questionnaire would not have answered it. In last year's survey, on the TRON Project as a whole, only a little more than half of the respondents answered questions about ITRON. In that sense, the ITRON-related questions last year had a response rate of 18.3%, so the 25% rate this year can be seen as an improvement.

This is the first year the questionnaires were distributed at a trade fair and made available on the Internet. Due to our lack of experience with these survey methods, the desired effectiveness was not achieved. In the case of the trade fair, the questionnaires were mixed in with a large number of other materials; and the response rate was not what we had initially expected.

1.3 Scope and items surveyed

Because of the nature of the survey, the questions were directed primarily at developers of embedded systems, who are either users or potential users of a real-time OS. The main items surveyed are as follows.

(1) Recently developed embedded systems

The survey asked about the application fields, system scale and real-time OS used in the three embedded systems most recently developed by the respondent.

(2) Problems with real-time OSs and basis for selection

(3) Awareness of ITRON-specification OS and related activities, etc.

(4) Advantages and disadvantages of an ITRON-specification OS

Persons using, developing or investigating the use of an ITRON-specification OS were asked about the advantages and disadvantages.

(5) Wishes for and opinions concerning the ITRON subproject

1.4 Occupations of the respondents

Approximately 80% of the respondents said they were in design or development, which means the survey reached its intended audience.

2. Analysis of Survey Results

2.1 Recently developed embedded systems

The respondents were asked about the application fields, CPU size, program size, and real-time OS used in the case of the three embedded systems most recently developed by them or their company.

(1) Application fields of recently developed embedded systems

Consumer-oriented products (home appliances, audio/visual, entertainment and education, personal information appliance, communication equipment (terminal)) and industrial equipment (plant control, factory automation, commercial equipment) accounted for around 30 percent each, with the remaining 40 percent going to other kinds of industrial systems. These can be seen as appropriate results when it is considered that they represent the number of systems developed, not the number produced, given that consumer products tend to be produced in much larger volumes than industrial systems.

(2) CPU used in recently developed embedded systems

In recently developed embedded systems, 32-bit CPUs occupied the largest share at around 45 percent, followed by 16-bit and 8-bit models, indicating that 32-bit processors have become the mainstream choice even in the embedded control field. Use of 64-bit processors is still very rare, and 4-bit chips accounted for a mere 2 percent of the total.

It should be noted, however, that these results are skewed somewhat toward high-end processors for the following reasons, and do not exactly mirror the actual picture overall. For one thing, the survey asked respondents to choose up to three of the most recently developed systems; and in a survey on a real-time OS, the tendency would be to choose systems of a scale that would likely use a real-time OS. Also, if one system uses several processors, the respondents would tend to focus on the main processor in their response, so that in fact, 4-bit processors are probably in wider use than these results suggest.

(3) Program size in recently developed embedded systems

Program sizes were divided rather evenly into the four categories of 64 KB or less, 64 KB to 256 KB, 256 KB to 1 MB, and 1 MB or more. That a quarter of the systems had a program size of 1 MB or more attests to the growing scale of embedded software. This is a trend that bears close watching.

The graph comparing program size to the size of the CPU confirms the trend toward using larger programs with the more powerful CPUs.

In systems using 32-bit CPUs, which were the majority, more than 40 percent had a program size of 1 MB or more, indicating again the trend toward large-scale embedded systems. Still, a quarter of these systems used programs of 256 KB or less, showing that a significant number of systems were able to keep the program size small even with powerful processors.

(4) OS used in recently developed embedded systems

The overall trend in OS use shows that 28.6 percent of the systems reported on use no OS, 25.6 percent use an ITRON-specification OS (broken down to 15.6% commercial ITRON-specification OS and 10.0% in-house ITRON-specification OS), 20.7 percent use an original in-house OS, and 25.1 percent of the systems use a non-ITRON commercial OS. Of the systems that use an OS, a third use an ITRON-specification OS; and of the commercial OSs used, around 40 percent are ITRON-specification OSs.

The breakdown of non-ITRON commercial OSs used is 6.2 percent for VxWorks and other Wind River Systems products, 4.1 percent for Microsoft OSs such as MS-DOS and Windows, 1.9 percent each for Integrated Systems (ISI) OSs such as pSOS and for Accelerated Technology OSs such as Nucleus Plus, and 1.5 percent each for a CTRON-specification OS and for Microware Systems OSs such as OS-9. All other OS brands were less than 1 percent each.

Looking at the relation of CPU size to OS used, the following trends emerge. First of all, it comes as no surprise that there were no examples of OS use with a 4-bit processor. Around a quarter of 8-bit systems use an OS, of which approximately half (i.e., one eighth of the total number of 8-bit systems) use an in-house original and a third (a twelfth of the total) use an ITRON-specification OS, with the remainder using other commercial OSs. Of 16-bit systems, around a quarter use no OS and another quarter use in-house original OSs, these two categories making up half the total. When it comes to 32-bit systems, approximately 10 percent use no OS and just under 20 percent use an in-house original OS, together accounting for less than a third of the total. In spite of this difference, the rate of ITRON-specification OS use for both 16-bit and 32-bit systems is around 30 percent, showing the wide use of ITRON-specification OS regardless of CPU size. The difference between 16-bit and 32-bit systems in the ratio of systems that use no OS or an in-house original OS corresponds to the difference in the use ratio of other commercial OSs.

Program size relates to OS use as follows. Of systems with a program size of 64 KB or less, approximately three-fourths do not use an OS and 8 percent use an ITRON-specification OS. This trend very closely matches that of 8-bit CPUs. In all other cases the rate of ITRON-specification OS use is around 30 percent, showing that the use of an ITRON-specification OS is not dependent on program size. In the case of in-house original OSs (with the exception of program sizes smaller than 64 KB), the use rate tends to go down as program size goes up; but in all cases the range is between 20 and 30 percent usage. The percentage of systems with no OS goes down as program size increases, and in turn the rate of other (non-ITRON) commercial OS use goes up. Of systems with a program size of 1 MB or larger, 43 percent use a non-ITRON commercial OS.

Finally, the relation of application field to OS use was analyzed. The graph arranges applications in the order of ITRON-specification OS use, in descending order from left to right. Some application fields have only a small sample size (the modulus in determining the ratio of use); but the general trend shown here is that ITRON-specification OSs are used especially in consumer-oriented systems, with over a 30 percent share in each of the fields of Home appliances, Audio/visual equipment, Entertainment, education, Personal information appliances, and Communication equipment (terminals). The wide use of ITRON-specification OSs in consumer products is seen in these results.

In each of the fields of Personal computer peripheral/office equipment, Industrial control, factory automation, commercial equipment, and Medical equipment, an ITRON-specification OS is used in from 20 to 25 percent of the systems, and commercial OSs other than ITRON have a high use rate.

In the field of Communication equipment (network equipment), the ITRON-specification OS share is only 15 percent, but 13 percent of the systems use a CTRON-specification OS, making the overall share of TRON-specification OSs in this field close to 30 percent. In the field where the ITRON-specification OS has the lowest use rate, Transportation equipment, the systems that do not use an OS make up 58 percent of the total, and the use of in-house original OSs is also relatively high, indicating that this field may be a difficult one for application of any general-purpose OS, not just ITRON.

2.2 Real-time OS problems and selection criteria

All the survey subjects were shown lists of possible problems with using a real-time OS and of OS selection criteria, from which they were asked to choose the items most important and next-most important. If they chose "Other" they could write in any other problems or criteria. The two graphs below are for "Single response" and "Multiple responses" respectively, with the first showing the totals only for items ranked as most applicable, and the latter showing the totals for items ranked as either most important or next-most important.

(1) Problems with using a real-time OS

The question was phrased, "What are the problems you (or your company) face in using a real-time embedded OS?" The answer chosen most often was, "An absence or shortage of staff familiar with the technology," which more than half chose as a problem and 28 percent selected as the most important problem.

In the single-response totals, the next most common problems selected were, "Major differences in OS specifications, making it hard to switch," "Performance and functions do not meet our requirements," "OS size or resources used are too large," "Lack of a development environment and tools," and "Cost is too high," each of which received 13 percent of the votes for most important problem. If the first two of these are combined under the issue of OS functions and performance not matching the application needs, this would be the second biggest problem.

When multiple responses are looked at, of those issues that were tied at 13 percent, "Lack of a development environment and tools" emerges with around 40 percent of the total, vaulting it ahead of the other items. The issue of development environment and tools is not the biggest problem but can be seen as the next-most important problem.

When those who chose "Lack of a development environment and tools" were asked to name the specific tools lacking, more than half listed tools for debugging. A common complaint was the lack of a task-aware debugger. Many respondents also listed in-circuit emulators and tools for performance evaluation and validation (including real-time performance validation). Other needs expressed in this connection included tools for support at the design and upstream processes, and tools running on a personal computer. Altogether the responses pretty much covered the range of tools for software development.

(2) Real-time OS selection criteria

In the single-response results, the most important selection criterion was given as "Performance and functions match our requirements," which received 36% of the votes. This was followed by, "Has a proven track record in our company" and "Is widely used in the industry," both focusing on the history of OS adoption and together making up 24% of the total. Vendor support, by contrast, was listed by only 2 percent of the respondents. It would be interesting to find out whether this result means vendor support is not considered important or whether vendor support is adequate and not seen as an issue.

Looking at the results when multiple responses are considered, history of adoption receives less overall weight, with more weight on two criteria that are under 10 percent each in the single-response results, namely, "Good development environment and tool support" and "Low cost," which come in second and third.

2.3 Awareness of ITRON-specification OS and related activities

(1) Awareness of the ITRON-specification OS

The responses "Have used or developed an ITRON-specification OS" and "Have investigated or studied its use" together made up around two-thirds of the responses, whereas "Was not aware of ITRON" got only one response, indicating a considerably high awareness of the ITRON specifications. The 39 percent response rate for "Have used or developed an ITRON-specification OS" would appear to be congruous with the ITRON-specification OS use rate of approximately 25 percent. As for those who responded, "Have heard about ITRON," it will be necessary to carry out more intensive promotional efforts by having them investigate and study the ITRON specifications.

(2) Awareness of ITRON-related activities

When asked to check ITRON-related activities of which they were aware, two out of three respondents checked, "Booths or presentations at trade shows and exhibitions." Apart from that response, the next most common item, awareness of the ITRON Technical Committee, received only around 35 percent of the responses, while the other items (except for a few items with extremely low awareness) got between 20 and 30 percent. Considering that 39 percent of the respondents said they had experience with using or developing an ITRON-specification OS, these results are rather low, indicating the need for public relations efforts to increase awareness of ITRON-related activities.

2.4 Advantages and disadvantages of the ITRON-specification OS

Those respondents who answered the question at 2.3 (1) that they have used, developed, investigated or studied the ITRON-specification OS were asked about the advantages and disadvantages of this OS. They were shown a list of items and asked to pick the most important and next-most important. As in 2.2, the graphs are for "single response" and "multiple response" results.

(1) ITRON-specification OS advantages

The graph of single-response results shows that "The specifications are easy to understand" and "Small OS size and resource use" each received more than 20 percent of the votes, indicating that the respondents have a clear understanding of the ITRON-specification OS advantages. The large number of respondents who chose, "The specifications are easy to understand" in particular shows the success of the ITRON specification design policy that emphasizes engineer training. Many respondents also chose, "Supports a wide range of chips," which indicates they appreciate that the ITRON specifications have been implemented on many different processors. On the other hand, only a very small number of votes went to "Good development environment and tools." When multiple responses are analyzed, "Low cost" receives a relatively high score.

(2) ITRON-specification OS disadvantages

In single-response results, more than a third of the respondents chose "Lack of a development environment and tools" as the biggest disadvantage, reaffirming a problem that is often pointed out. Next in frequency was (by a slim margin), "No notable disadvantages," with 11 percent of the votes. Looking at multiple-response results, relatively high shares were received by, "Too many implementation-dependent parts making it hard to port" and "Lack of engineers familiar with it," which got only ten percent each of the single-response votes. These results would indicate the increased necessity for standardization to improve software portability.

2.5 Wishes for and opinions on ITRON-related activities

Respondents were asked to select from a list of areas those in which they would like the ITRON subproject to become involved. The most common request, cited by 55 percent of the respondents, was for "Standardization of interfaces with development environments (esp. debuggers)"; next were, "Interface standards for software components (middleware)" and "Network support" with more than 40 percent each, followed by, "C++ and Java language binding standards" and "Free ITRON-specification OS" in the 30 percent range. Given the supposed lack of engineers who can work with a real-time OS, surprisingly few votes went to, "Holding training seminars."

There was space on the questionnaire for writing in additional requests and opinions on ITRON activities. Since the total number of such responses was small, it is difficult to discern any trends; but a number of respondents requested enhanced PR and advertising, and development of a more compact OS specification.

2.6 Summary

One trend that can be culled from the survey results on recently developed systems is the growing scale of embedded systems. Of the systems that use an OS, more than a third use an ITRON-specification OS, far more than any other OS specification, which confirms the place of ITRON as an industry-standard specification in Japan.

The advantages and disadvantages cited for the ITRON-specification OS are nearly in line with those the ITRON Technical Committee is already aware of, confirming the correctness of that awareness. In particular, the very large number of respondents who cited the inadequacy of the development environment and tools as an ITRON-specification OS disadvantage points up once again the strong need for improvements in this area. The results on ITRON awareness demonstrate a rather high level of familiarity with the ITRON-specification OS itself, but insufficient awareness of related activities, showing the need for more effective promotional efforts.

Another result emerging is that nearly 30 percent of the systems were built without an OS. In small-scale systems the inability to employ an OS can be seen as unavoidable, but there are also cases where no OS was used in large-scale systems, suggesting the need to expend more effort promoting the spread of real-time OSs. One of the main reasons cited for not using an OS is the lack of skilled engineers. From this result we can see the importance of training engineers in the use of a real-time OS.

Whereas past surveys have covered the entire TRON Project, this year's survey was limited to the ITRON-specification OS; moreover, the questions were completely revised in order to get an accurate idea as to what kinds of operating systems are used in embedded systems. Unfortunately, this makes it impossible to compare these results with those of previous years. Still, we feel the survey was able to produce quite accurate results overall, and we look forward to the results of similar surveys in coming years.

Reference

• Questionnaire Form on ITRON-specification OS Use Trends