Competitive intelligence

Historic development

Competitive intelligence literature can be exemplified by the bibliographies that were published in the Strategic Consortium of Intelligence Professionals' now discontinued academic journal The Journal of Competitive Intelligence and Management. Although elements of organizational intelligence collection have been a part of business for at least a century, the history of competitive intelligence as a discrete topic began in the U.S. in the 1970s, although the literature on the field pre-dates this time by at least several decades.

In 1980, Michael Porter published the study Competitive-Strategy: Techniques for Analyzing Industries and Competitors which is viewed as the foundation of modern competitive intelligence. This has since been extended most notably by Craig Fleisher and Babette Bensoussan, who through several popular books on competitive analysis described 48 competitive intelligence analysis techniques to the practitioner's tool box.

Although Michael Porter first discussed the importance of monitoring and analyzing competitors, the discipline became a separate branch of business strategy in 1985, when Leonard Fuld published a book dedicated to competitor intelligence. However, the institutionalization of competitive intelligence as a formal activity among American corporations can be traced to 1988, when Ben and Tamar Gilad published the first organizational model of a formal corporate competitive intelligence function, which was then adopted widely by US companies. The first professional certification program (CIP) was created in 1996 with the establishment of The Fuld-Gilad-Herring Academy of Competitive Intelligence in Cambridge, Massachusetts - now known as the Academy of Competitive Intelligence. Within Europe, the Institute for Competitive Intelligence, based in Germany offers similar training and certification.

Also in 1986, the "Society of Competitor Intelligence Professionals" (SCIP) was founded - later changing names to become the "Society of Competitive Intelligence Professionals" (reflecting that the focus for CI was not just competitors). SCIP was initially an independent non-profit but ran out of money and in 2009 was merged with the Frost & Sullivan Institute . SCIP grew in the late 1990s to around 6,000 members worldwide, mainly in the United States and Canada, but with large numbers especially in the UK and Australia. SCIP was then renamed Strategic & Competitive Intelligence Professionals to emphasize the strategic nature of the subject, and also to refocus the organization's general approach, while keeping the existing SCIP brand name and logo. In April 2023, SCIP was renamed again to the Strategic Consortium of Intelligence Professionals. The renaming was intended to reflect the organization’s expanded global scope, encompassing practitioners in competitive, strategic, economic, and social intelligence.

A number of efforts have been made to discuss the field's advances in post-secondary (university) education, covered by several authors including Blenkhorn & Fleisher, Fleisher, Fuld, Prescott, and McGonagle. Although the general view would be that competitive intelligence concepts can be readily found and taught in many business schools around the globe, there are still relatively few dedicated academic programs, majors, or degrees in the field, a concern to academics in the field who would like to see it further researched. These issues were widely discussed by over a dozen knowledgeable individuals in a special edition of the Competitive Intelligence Magazine that was dedicated to this topic. In France, a Specialized Master in Economic Intelligence and Knowledge Management was created in 1995 within the CERAM Business School, now SKEMA Business School, in Paris, with the objective of delivering a full and professional training in Economic Intelligence, although this is no longer offered. A Centre for Global Intelligence and Influence was created in September 2011 in the same institution.

Practitioners and companies regard professional accreditation as important in this field. In 2011, SCIP recognized the Fuld-Gilad-Herring Academy of Competitive Intelligence's CIP certification process as its global, dual-level (CIP-I and CIP-II) certification program.

Global developments have also been uneven in competitive intelligence. Several academic journals, particularly the Journal of Competitive Intelligence and Management in its third volume, provided coverage of the field's global development. For example, in 1997 the École de guerre économique (School of economic warfare) was founded in Paris, France. It is the first European institution which teaches the tactics of economic warfare within a globalizing world. In Germany, competitive intelligence was unattended until the early 1990s. The term "competitive intelligence" first appeared in German literature in 1997. In 1995, a German SCIP chapter was founded, which is now second in terms of membership in Europe. In 2004, the Institute for Competitive Intelligence was founded, which provides a postgraduate certification program for Competitive Intelligence Professionals. Japan is currently the only country that officially maintains an economic intelligence agency (JETRO). It was founded by the Ministry of International Trade and Industry in 1958.

Accepting the importance of competitive intelligence, major multinational corporations, such as Amazon, Microsoft, and Procter & Gamble have created formal competitive intelligence units. Organizations execute competitive intelligence activities not only as a safeguard to protect against market threats and changes, but also as a method for finding new opportunities and trends.

Organizations use competitive intelligence to compare themselves to other organizations ("competitive benchmarking"), to identify risks and opportunities in their markets, and to pressure-test their plans against market response (business wargaming), which enable them to make informed decisions.

One of the major activities involved in corporate competitive intelligence is use of ratio analysis, using key performance indicators (KPIs). Organizations compare annual reports of their competitors on certain KPIs and ratios, which are intrinsic to their industry. This helps them track their performance, vis-à-vis their competitors.

The actual importance of these categories of information to an organization depends on the contestability of its markets, its organizational culture, the personality and biases of its top decision-makers, and the structure of competitive intelligence reporting within the company.

Strategic intelligence (SI) focuses on the longer term, looking at issues affecting a company's competitiveness over the course of a couple of years. The actual time horizon for strategic intelligence ultimately depends on the industry and how quickly it's changing. The general questions that strategic intelligence answers are, ‘Where should we as a company be in X years?' and 'What are the strategic risks and opportunities facing us?' This type of intelligence work involves among others the identification of weak signals and application of methodology and process called Strategic Early Warning (SEW), first introduced by Gilad, followed by Steven Shaker and Victor Richardson, Alessandro Comai and Joaquin Tena, and others. According to Gilad, 20% of the work of competitive intelligence practitioners should be dedicated to strategic early identification of weak signals within a SEW framework.

Tactical Intelligence: the focus is on providing information designed to improve short-term decisions, most often related to the goal of growing market share or revenues. Generally, it is the type of information that a person would need to support the sales process in an organization. It investigates various aspects of product or product-line marketing.

With the right amount of information, organizations can avoid unpleasant surprises by anticipating competitors' moves and decreasing response time. Examples of competitive intelligence research is evident in daily newspapers, such as The Wall Street Journal, Business Week, and Fortune. Major airlines change many of their fares daily in response to competitors' tactics. They use information to plan their own marketing, pricing, and production strategies.

Resources, such as the Internet, have made gathering information on competitors easy. Analysts can discover future trends and market requirements. However, competitive intelligence is much more than this, as the ultimate aim is to lead to competitive advantage. As the Internet is mostly public domain material, information gathered is less likely to result in insights that will be unique to the company. There is a risk that information gathered from the Internet will be misinformation and mislead users, so competitive intelligence researchers are often wary of using such information.

As a result, although the Internet is viewed as a key source, most competitive intelligence professionals should spend their time and budget gathering intelligence using primary research—networking with industry experts, from trade shows and conferences, from their own customers and suppliers, and so on. Where the Internet is used, it is to gather sources for primary research as well as information on what the company says about itself and its online presence (in the form of links to other companies, its strategy regarding search engines and online advertising, mentions in discussion forums and on blogs, etc.). Online subscription databases and news aggregation sources, which have simplified the secondary source collection process, are also used.

Use

A 1998 study by the Futures Group, a Glastonbury, Connecticut-based consulting firm, indicates that 82% of companies with annual revenues exceeding $10 billion and 60% of those with annual revenues exceeding $1 billion now have an organized intelligence system.

While most larger companies have specific departments devoted to Commercial Intelligence, mid-sized companies tend to hire Commercial Intelligence firms, and smaller business owners are more likely to do it themselves.

The technical advances in massively parallel processing offered by the Hadoop "big data" architecture has allowed the creation of multiple platforms for named-entity recognition such as the Apache Projects OpenNLP and Apache Stanbol. The former includes pre-trained statistical parsers that can discern elements key to establishing trends and evaluating competitive positions and responding appropriately. Public information mining from SEC.gov, Federal Contract Awards, social media, vendors, and competitor websites now permits real-time counterintelligence as a strategy for horizontal and vertical market expansion and product positioning. This occurs in an automated fashion on massive marketplaces such as Amazon.com and their classification and prediction of product associations and purchase probability.

Outsourcing

Outsourcing has become a significant business for competitive intelligence professionals. There are many different companies in this field, including market research and consulting firms.

Primary and Secondary Intelligence

Primary Sources of Intelligence

Primary sources of Commercial Intelligence include:

• Speeches by executives about future strategies, company tactics, and company literature distribute at trade shows, conferences, and other events.
• All forms of communication media such as TV, radio, business magazines, etc.
• Annual reports.
• Physical observations of activities.
• Conversations with internal sources known to be knowledgeable about the competitor's operations and decision making
• Interviews with former company employees
• Discussions with the company's consultants or agencies

Secondary Sources of Intelligence

Secondary sources of CI include:

• Articles and reviews created by someone outside the company concerning the company.
• Books about the company and/or the companies in the industry.
• Special studies, research papers, and analyst reports.

Intelligence from both classifications can be found within the scope of open source intelligence (OSINT).

The Commercial Intelligence Cycle

Efficient and successful CI is a constant cycle that consists of 5 steps:

1. Planning; deliberating with decision makers to discover their intelligence needs.
2. Gathering; collecting activities that are conducted legally and ethically.
3. Analysis; interpreting all the information gathered and assembling the suggested actions.
4. Dissemination; presenting the findings to the decision makers
5. Feedback; studying the decision makers’ responses and understanding their need for continued intelligence.

Similar fields

Competitive intelligence has been influenced by national strategic intelligence. Although national intelligence was researched 50 years ago, competitive intelligence was introduced during the 1990s. Competitive intelligence professionals can learn from national-intelligence experts, especially in the analysis of complex situations. Competitive intelligence may be confused with (or seen to overlap) environmental scanning, business intelligence, and market research. Craig Fleisher questions the appropriateness of the term, comparing it to business intelligence, competitor intelligence, knowledge management, market intelligence, marketing research, and strategic intelligence.

Fleisher suggests that business intelligence has two forms. Its narrow (contemporary) form is more focused on information technology and internal focus than competitive intelligence, while its broader (historical) definition is more inclusive than competitive intelligence. Knowledge management (KM), when improperly achieved, is seen as an information-technology driven organizational practice relying on data mining, corporate intranets and mapping organizational assets to make it accessible to organization members for decision-making. Competitive intelligence shares some aspects of knowledge management; they are human-intelligence- and experience-based for a more-sophisticated qualitative analysis. Knowledge management is essential for effective change. A key effective factor is a powerful, dedicated IT system executing the full intelligence cycle.

Market intelligence (MI) is industry-targeted intelligence developed in real-time aspects of competitive events taking place among the four Ps of the marketing mix (pricing, place, promotion and product) in the product (or service) marketplace to better understand the market's attractiveness. A time-based competitive tactic, market intelligence is used by marketing and sales managers to respond to consumers more quickly in the marketplace. Fleisher suggests it is not distributed as widely as some forms of CI, which are also distributed to non-marketing decision-makers. Market intelligence has a shorter time horizon than other intelligence areas, and is measured in days, weeks, or (in slower-moving industries) months.

Market research is a tactical, method-driven field consisting of neutral, primary research of customer data (beliefs and perceptions) gathered in surveys or focus groups, and is analyzed with statistical-research techniques. Competitive intelligence draws on a wider variety (primary and secondary) of sources from a wider range of stakeholders (suppliers, competitors, distributors, substitutes and media) to answer existing questions, raise new ones and guide action.

Ben Gilad and Jan Herring lay down a set of prerequisites defining competitive intelligence, distinguishing it from other information-rich disciplines such as market research or business development. They show that a common body of knowledge and a unique set of tools (key intelligence topics, business war games and blindspots analysis) distinguish competitive intelligence; while other sensory activities in a commercial firm focus on one segment of the market (customers, suppliers or acquisition targets), CI synthesizes data from all high-impact players (HIP).

Gilad later focused his delineation of CI on the difference between information and intelligence. According to him, the common denominator among organizational sensory functions (whether they are called market research, business intelligence or market intelligence) is that they deliver information rather than intelligence. Intelligence, says Gilad, is a perspective on facts rather than the facts themselves. Unique among corporate functions, competitive intelligence has a perspective of risks and opportunities for a firm's performance; as such, it (not information activities) is part of an organization's risk-management activity.

Ethics

Ethics has been a long-held issue of discussion among competitive intelligence practitioners. The questions revolve around what is and is not allowable in terms of competitive intelligence activity. Several scholarly treatments have been generated on this topic, most prominently addressed through Strategic Consortium of Intelligence Professionals publications. The book Competitive Intelligence Ethics: Navigating the Gray Zone provides nearly twenty separate views about ethics in competitive intelligence, as well as another 10 codes used by various individuals or organizations. Combining that with the over two dozen scholarly articles or studies found within the various competitive intelligence bibliographic entries, it is clear that no shortage of study has gone into better classifying, understanding, and addressing CI ethics.

Competitive information may be obtained from public or subscription sources, from networking with competitor staff or customers, disassembly of competitor products or from field research interviews. Competitive intelligence research is distinguishable from industrial espionage, as competitive intelligence practitioners generally abide by local legal guidelines and ethical business norms.

Misrepresentation

Misrepresentation in CI is a form of social engineering. It involves falsely identifying yourself and deceiving people into providing personal information that they would not normally disclose. Misrepresentation is the most common issue that divides CI practitioners and raises numerous complex ethical concerns.

The most common types of misrepresentation include:

a. Excluding specific details in one's identity: Lying about one's identity to gain access to certain information is known to be unethical amongst CI practitioners. However, there is a thin line between lying about one's identity and omitting certain details about it. Some practitioners believe that it is acceptable to deliberately skip over some details to obtain information.

For example: Person X is both a part-time student and a director of a large company. As part of a school assignment, X is expected to collect information about a company that happens to be one of X's employer's biggest competitors.

In this case, it is believed by some CI practitioners that not revealing the fact that X works for the company in addition to being a part-time student is completely ethical. In other words, X is misleading the competitor into thinking the information given to him will be used to complete a school assignment and nothing more, although, in reality, he would be gathering commercial intelligence to present to his employer.

However, these types of situations raise some ethical concerns because the competitor may not have provided certain information knowing X works for a direct competitor. The competitor was not lied to but deliberately misled in to give out specific information about the company.

b. Overhearing classified information: Some CI practitioners believe that if you overhear a competitor's conversation in a public venue, then the information discussed that was previously classified becomes ethical for you to use to your advantage.

They believe that it is not necessary to reveal their identity since the conversation occurred in an open and public place, and it is the competitors’ fault that their information has been leaked because they should have been more aware of their surroundings and been more secure with their information. On the other hand, some argue that the CI practitioners are at fault because they deliberately planned on being in those surroundings in hopes to overhearing some information, which raises concerns about the ethics of the matter.

c. Not Disclosing true intent on how information will be used: Many CI practitioners are paid by their clients to get as much information about their competitors. To do so, they would survey the competitors and suggest to them that they are simply collecting industry information, all the while not revealing who they are working for, thus, not disclosing the true intent on how the information gathered is going to be used.

All 3 misrepresentation issues share a common theme, which is: “What is the intent of the CI practitioner?” Judging by a person's intent, we can conclude whether or not the situation is ethical or non-ethical.

Client Conflict

Client conflict means any situation in which a practitioner is faced with a conflict of interest between a former and current client. The practitioner must attempt to determine how to act in the best interests of several clients in the same or substantially related matter.

This type of situation usually arises with CI practitioners who are consultants. However, a clear solution is obvious; the practitioners agree that they should never service competing clients at the same time to keep classified information secure. In addition, another conflict may arise in the CI consultant and client relationship. The client usually hires the consultant to gather as much information about their competitors.

However, there is an ethical issue concerning the length in which a consultant would go to retrieve the required information; how much is too much? When the consultant ignores the cost at which they are seeking this information, they tend to stumble upon various ethical issues, and it is these ethical issues that clients don't want to be associated with that push them to hire a CI consultant.

Cases of Organizations Accused of Unethical CI Practices

Oracle Corporation

In 2000, Oracle Corporation hired a detective agency to investigate two research groups that supported Microsoft during the antitrust trial. After attempting to buy garbage, the agency discovered that those two research groups were falsely identifying themselves as independent advocacy groups when they were in reality funded by Microsoft.

Oracle and the hired detective agency may have acted in questionable matters to obtain that information, but they believe that the outcome justifies their actions. Larry Ellison, Oracle's chairman, says: “I feel very good about what we did. […] Maybe our investigation organization may have done things unsavory, but it's not illegal. We got the truth out.”

WestJet

In May 2004, Air Canada filed a $5 million corporate espionage lawsuit against WestJet after discovering that their rival had hacked into Air Canada's internal employee-only website over 243,000 times in a period of 10 months. WestJet used the confidential information it found to rearrange its load schedules as well as adjust routes and it significantly reduced service from the Hamilton airport and increased service in Toronto.

WestJet later issued a rare apology to its rival and Robert Milton, the chief executive of parent ACE Aviation Holdings Inc, admitting that its actions of online snooping "were both unethical and unacceptable" and ended up paying AirCanada $15.5 million in legal fees as well as donating $10-million to children's charities in the names of both airlines. Moreover, Hill, one of the founders of WestJet later resigned from his position as vice-president of strategic planning in July 2004.

Hewlett-Packard

In 2005, HP filed a lawsuit against its former VP, Karl Kamb, for $100 million, claiming Kamb had betrayed the company and appropriated its trade secrets as well as its money to start up his own flat-panel-TV company, as he was working on the TV project for HP. In January 2007.

Kamb filed a countersuit. Not only did he deny stealing trade secrets but also claimed that HP was aware of what he had done with its money.

He claimed that HP had asked him to gather classified information on Dell, whose entry into printers had threatened HP's most profitable line of business. In 2002, the CI unit hired Katsumi Iizuka, former president of Dell Japan until 1995, selling HP information on Dell's plans to enter the printer business. The information they gathered revolved around printer models, specifications, terms, and prices, many months before their launch. Furthermore, HP was accused of pretexting, which is the act of lying about one's identity to obtain privileged data and information, to obtain Kamb's private phone records.

Society of Competitive Intelligence Professionals

The Society of Competitive Intelligence Professionals is a not-for-profit association whose 7,000 members conduct competitor research and analysis for large and small companies, and help manage planning competitive strategy.

Established in 1986 in Washington DC, it is currently headquartered in Alexandria, Virginia. It focuses on enhancing the success of its members through education, leadership, support, and networking. Since 1986, SCIP has developed drastically and globally, and has, today, chapters all around the world as well as alliance partnerships with numerous independent affiliate organizations.

On July 8, 2010, the SCIP Board of Directors voted to officially change the 25-year-old non-profit organization's name from "Society of Competitive Intelligence Professionals" to "Strategic and Competitive Intelligence Professionals".

Chapters Overview

SCIP Chapters are an efficient way for members to build relationships within the competitive intelligence discipline in their area. Its events are open for anyone to attend, become involved, and reap the benefits of programming and networking. The SCIP offers many opportunities to its members including a range of educational and networking experiences with leading industry experts, as well as opportunities to increase knowledge by gaining access to unique practices, tools, and publications.

In addition, the SCIP offers Collegiate Chapters for students who show interest in the world of commercial and competitive intelligence. Participant students in the Collegiate Chapter are offered training and education programs that represent unique opportunities for professional advancement. They are also benefitting in terms of leadership skills, career development, and indispensable CI know-how. Also, the SCIP awards mentoring programs and scholarships to students who are deemed prominent, promising, and capable business students.

Ethics

Economic espionage represents a failure of competitive intelligence, which uses open sources and other forms of ethical inquiry. Importantly, SCIP mandates that all this be done within a strictly ethical framework, which is why it has agreed upon a code of ethics:

• To continually strive to increase the recognition and respect of the profession.
• To comply with all applicable laws, domestic and international.
• To accurately disclose all relevant information, including one's identity and organization, before all interviews.
• To avoid conflicts of interest in fulfilling one's duties.
• To provide honest and realistic recommendations and conclusions in the execution of one's duties.
• To promote this code of ethics within one's company, with third-party contractors, and within the entire profession.
• To faithfully adhere to and abide by one's company policies, objectives, and guidelines.