Entries Tagged 'Search' ↓

I attended the inaugural Enterprise Search Europe in London October 24-25 and was impressed with a few things. Firstly the attendance was stronger than I thought, perhaps around 100, although that includes sponsors’ attendees. There were half a dozen main sponsors and two vendors I spoke to said they wanted to sponsor it but were too late – so there is demand there for such an event. Chairman Martin White of Intranet Focus did an excellent job of keeping things moving along and stepped in to fill a gap in the program caused by last-minute absentees. He’s summed up the whole conference himself here.

I was on two panels, one more formal than the other. I was responding to the thoughts of Funnelback’s chief scientist David Hawking, who talked of current research in the enterprise search area and contrasted it with the wealth of research done in Web search. I pointed out that if some organizations such as TREC made its output actually usable by vendors (who aren’t permitted to say where they come in TREC rankings) then it would get much more support.

In the evening in the Hand & Flower pub across the road from the hotel for the latest Enterprise Search Meetup London I was one of two permanent panelists (Laura Wilber of Exalead being the other), with a goldfish bowl set-up where anyone wanting to ask a question had to joint the panel’s spare seats to do it from there. It made for some pretty lively discussions about what the next big things might be impacting enterprise search (big data being one of them, of course), helped by the  beer & wine paid for by Exalead.

The evening in the pub might have been livelier than the panel at the conference itself, but the themes were similar, namely that enterprise search as a use case and value proposition is dead; people don’t buy enterprise search engines to ‘find stuff’ within their organization anymore. There has to be a more tangible use case, such as electronic discovery, or fraud detection and the like. I’ve written about this recently.

Two other analysts were at the conference – Alan Pelz-Sharpe of RealStoryGroup and Mike Davis of Ovum. Alan has penned his forthright thoughts here and Mike and I talked enough to know that the three of us – and plenty of others at the event that weren’t analysts but actual users – were thinking similar thoughts. So despite the fact that the conference is called Enterprise Search Europe and most people seem to think that enterprise search as a concept is redundant, I would recommend the event for next year, by when I suspect it may be called something else.

As a follow-up to Matt’s post last week showing where he’ll be speaking during Q4, here’s some more updates of other in the information management team speaking at various events this quarter.

First up I’m chairing and speaking at IQPC’s Enterprise Information Management Exchange in London on October 10-11. I’m speaking to a mainly C-level end user audience about information risk management, moderating a panel on  how to make the most of your information assets and brushing off my MC-ing skills to keep the whole show moving along.

Next up I’ll be back in NYC at Text Analytics World giving a slightly shorter version of a similar presentation on October 19 (which I’ll be refining and also presenting at Predictive Analytics World in London on November 30).

On October 24 I’m on the opening panel of Enterprise Search Europe, discussing the issues brought up by the keynote presentation by Funnelback’s David Hawking, among other things

On October 27 David Horrigan will be attending Guidance Software’s Federal Summit in Washington, DC where he’ll be moderating a panel called e-Discovery in the cloud. This is an invite-only one being handled by Guidance, so I don’t have a link unfortunately.

Into November and I’ll be attending the e-Discovery and e-Investigations Forum in London on November 10. There I’ll be discussing the choice available to end users in e-Discovery in a session called: A buyers guide to navigating the info management and e-discovery technology marketplace.’

The following week I’ll be in Munich at IQPC’s Information Retention & e-Discovery Exchange where I’m sitting on a couple of panels – one on social media in e-Discovery and another on technology in this area.

Finally this quarter  Kathleen Reidy will be attending Gilbane’s annual gathering of enterprise content management mavens where she’s moderating a panel entitled ‘Get Ready for Big Data.’

We hope to see some of you at one or more of these events in Q4.

Customers of The 451 Group would have seen my report on the enterprise search market published September 15. If you are a client, you can view it here. I thought it would be useful to provide a condensed version of the report to a wider audience as I think the market is at an important point it in its development and it merits a broader discussion.

The enterprise search market is morphing before our eyes into something new. Portions of it are disappearing, and others are moving into adjacent markets, but a core part of it will remain intact. A few key factors have caused this, we think. Some are historical, by which we mean they had their largest effect in the past, but the ongoing effect is still being felt, whereas the contemporary factors are the ones that we think are having their largest impact now, and will continue to do so in the short-term future (12-18 months).

Historical factors

• Over-promising and under-delivery of intranet search between the last two US recessions, roughly between 2002 and 2007, resulting in a lot of failed projects.
• A lack of market awareness and understanding of the value and risk inherent in unstructured data.
• The entrance of Google into the market in 2002.
• The lack of vision by certain closely related players in enterprise content management (ECM) and business intelligence (BI).

Contemporary factors

• The lack of a clear value proposition for enterprise search.
• The rise of open source, in particular Apache Lucene/Solr.
• The emergence of big data, or total data.
• The social media explosion.
• The rapid spread of SharePoint.
• The acquisitive growth of Autonomy Corp.
• Acquisition of fast-growing players by major software vendors, notably Dassault Systemes, Hewlett-Packard and Microsoft.

The result of all this has been a split into roughly four markets, which we refer to as low-end, midmarket, OEM and high-end search-based applications.

Entry-level search

The low-end, or entry-level, enterprise search market has become, if not commodified, then pretty close to it. It is dominated by Google and open source. Other commercial vendors that once played in it have mostly left the market.

The result is that potential entry-level enterprise search customers are left with a dichotomy of choices: Google’s yellow search appliances that have two-year-term licenses and somewhat limited configurability (but are truly plug-and-play options) on the one hand, and open source on the other. It is a closed versus a very open box, and they have different and equally enthusiastic customer bases. Google is a very popular department-level choice, often purchased by line-of-business knowledge workers frustrated at obsolete and over-engineered search engines. Open source is, of course, popular with those that want to configure their search engine themselves or have a service provider do it and, thus, have a lot of control over how the engine works, as well as the results it delivers. Apache Lucene is also part of many commercial, high-end enterprise search products, including those of IBM.

Midmarket search

Mid-market search is a somewhat vague area, where vendors are succeeding in deals of roughly $75,000-250,000 selling intranet search. This area has thinned out as some vendors have tried to move upmarket into the world of search-based applications, but there are still many vendors making a decent living here. However, SharePoint has had a major effect on this part of the market, and if enterprises already have SharePoint – and Microsoft reckons more than 70% have at least bought a license at some point already – then it can be tough to offer a viable alternative. However, if SharePoint isn’t the main focus, then there is still a decent business to be had offering effective enterprise search, often in specific verticals, albeit without a huge amount of vertical customization.

OEM

The OEM search business has become a lot more interesting recently, in part due to which vendors have left it, leaving space for others. Microsoft’s acquisition of FAST in early 2008 meant one of the two major vendors at the time had essentially left the market entirely, since its focus moved almost entirely to SharePoint, as we recently documented. The other major OEM vendor at the time was Autonomy, and while it would still consider itself to be so, we think much of its OEM business, in fact, comes from document filters, rather than the OEMing of the IDOL search engine. Autonomy would strongly dispute that, but it might be moot soon anyway – it now looks as if it will end up as part of Hewlett-Packard following the announcement of its acquisition at a huge valuation, on August 18.

Those exits have left room for the rise of other vendors in the space. Key markets here include archiving, data-loss prevention and e-discovery. Many tools in these areas have old or quite basic search and text analysis functionality embedded in them, and vendors are looking for more powerful alternatives.

Search-based applications

The high end of the enterprise search market has become, in effect, the market for search-based applications (SBA) – that is, applications that are built on top of a search engine, rather than solely a relational database (although they often work alongside a database). These were touted back in the early 2000s by FAST, but it was too early, and FAST was too complex a set of tools to give the notion widespread acceptance. But in the latter part of the last decade and this one, SBAs have emerged as an answer to the problem of generic intranet search engines getting short shrift from users dissatisfied that the search engines don’t deliver what they want, when they want it.

Until recently, SBAs have mainly been a case of the vendors and their implementation partners building one-off custom applications for customers. But they are now moving to the stage where out-of-the-box user interfaces are being supplied for common tasks. In other words, it’s maturing in a similar way to the application software industry 20 years ago, which was built on top of the explosion in the use of relational databases.

We’ve seen examples in manufacturing, banking and customer service, and one of the key characteristics of SBAs is their ability to combine structured and unstructured data together in a single interface. That was also the goal of earlier efforts to combine search with business-intelligence tools, which often simply took the form of adding a search engine to a BI tool. That was too simplistic, and the idea didn’t really take off, in part because search vendors hadn’t paid enough attention to structure data.

But SBAs, which put much more focus on the indexing process than earlier efforts, appear to be gaining traction. If we were to get to the situation where search indexes are considered a better way of manipulating disparate data types than relational databases, that would be a major shift (see big data). Another key element of successful SBAs is that they don’t look like traditional search engines, with a large amount of white space and a search bar in the middle of the screen. Rather, they make use of facets and other navigation techniques to guide users through information, or often simply to present the relevant information to them.

As I mentioned, there’s more in the full report, including more about specific vendors, total (or big) data and the impact of social media. If you’d like to know more about it, please get in touch with me.

Just after the HP call about its Q3 numbers and the deal, here’s my initial (very) quick take as it’s late here in London:

• This deal is about getting serious about software under Leo Apotheker. It gives HP a real information management story, greatly boosting its presence in the archiving, e-Discovery and enterprise search businesses.
• However, company cultures are not complementary, the HP way is a long way from the hyper-aggressive sales and marketing culture at Autonomy. Maintaining Autonomy as a separate entity run by Mike Lynch proves this and calls into question how much real synergy can be had from such a structure. I cannot see that being sustained.
• This instantly makes HP a bigger e-Discovery player than IBM or any of the major IT firms.
• Product overlap exists in document and records management but gets HP into the web content management and website optimization markets.
• Autonomy has resisted deals over the years as its market capitalization ballooned as it went on its own acquisition binge. Autonomy couldn’t have waited much longer as it would have grown too big to be swallowed by even the largest predator.
• At least Autonomy customers will now have a services organization to call on after they’ve bought the software. Customer support and after sales service has not been a strength of Autonomy.
• This leaves the FTSE 100 with just one software firm of note.

Our clients will have seen our preview of 2011 last week. For those that aren’t (yet!) clients and therefore can’t see the whole 3,500-word report, here’s the introduction, followed by the titles of the sections to give you an idea of what we think will shape the information management market in 2011 and beyond. Of course the IT industry, like most others doesn’t rigorously follow the wiles of the Gregorian calendar, so some of these things will happen next year while others may not occur till 2012 and beyond. But happen they will, we believe.

We think information governance will play a more prominent role in 2011 and in the years beyond that. Specifically, we think master data management and data governance applications will appear in 2011 to replace the gaggle of spreadsheets, dashboards and scorecards commonly used today. Beyond that, we think information governance will evolve in the coming years, kick-started by end users who are asking for a more coherent way to manage their data, driven in part by their experience with the reactive and often chaotic nature of e-discovery.

In e-discovery itself, we expect to see a twin-track adoption trend. While cloud-based products have proven popular, at the same time, more enterprises buy e-discovery appliances.

‘Big data’ has become a bit of a catchall term to describe the masses of information being generated, but in 2011 we expect to see a shift to what we term a ‘total data’ approach to data management, as well as the analytics applications and tools that enable users to generate the business intelligence from their big data sets. Deeper down, the tools used in this process will include new BI tools to exploit Hadoop, as well as a push in predictive analytics beyond the statisticians and into finance, marketing and sales departments.

SharePoint 2010 may have come out in the year for which it is named, but its use will become truly widespread in 2011 as the first service pack is release and the ISV community around it completes their updates from SharePoint 2007. However, we don’t think cloud-based SharePoint will grow quite as fast as some people may expect. Finally, in the Web content management (WCM) market – so affected by SharePoint, as well as the open source movement – we expect a stratification between the everyday WCM-type scenario and Web experience management (WEM) for those organization that need to tie WCM, Web analytics, online marketing and commerce features together.

• Governance family reunion: Information governance, meet governance, risk and compliance; meet data governance….
• Master data management, data quality, data integration: the road to data governance
• E-discovery post price war: affordable enough, or still too strategic to risk?
• Data management – big, bigger, biggest
• Putting the BI into big data in Hadoop
• The business of predictive analytics
• SharePoint 2010 gets real in 2011
• WCM, WEM and stratification

And with that we’d like to wish all readers of Too Much Information a happy holiday season and a healthy and successful 2011.

I knew that as soon as I wrote my updated look on which company Autonomy might buy next something would come along to diminish the value of all my hard work. 😉 I had expected it to be an acquisition, but instead Autonomy surprised me yesterday afternoon with an announcement that after being in talks regarding an acquisition for “several months,”

Recent developments within these talks have given rise to an additional opportunity that warrants further examination which could give rise to an acquisition process that exceeds our original planned time scale.

In other words, it’s not going to happen any time soon.

As an announcement it’s a bit opaque and strangely structured, coming as the second part of an announcement that also told announced its Capital Markets Day would be on Monday, November 29.

It could mean:

• Autonomy is attempting to buy part of a business and instead is now looking to expand into another part or perhaps all of the business. For that to be the case ther first part must have been quite a small deal as Autonomy’s kitty is limited to about $1bn in total.
• Autonomy has identified a different company that it wants to buy.
• Autonomy itself has had an offer to be acquired.

This is all of course speculation. But one thing’s a bit more certain: it will probably make a dent in the company’s Q4 results, which were expected to include a boost from an acquisition that now doesn’t look like happening. Whether that means the company will miss the revised expectations it set in the Q3 call, we’ll have to wait and see.

In our report we wondered whether the company’s health care announcement, including a new product called Auminence may be used as some sort of alternative kicker in the quarter, in lieu of an acquisition. As we said, the product came out of the blue, as do many Autonomy products and appears to us to be a repackaging of IDOL with some added diagnosis checklists on top.

The shares fell 6% yesterday after the announcement came out, all of that coming late in the day as the announcement was made at 3.51pm London time. Again, that’s slightly strange timing. The shares were creeping back up this morning.

UPDATE: Autonomy clarified its statement with this Q&A:

Would it be right to interpret from today’s announcement regarding the acquisition timetable (Update on Acquisition) that the deal you are negotiating has got larger?

No, the deal remains the same size. The statement clearly refers to the timescale.

So it’s the same company, it will just take longer, which makes most sense, I guess.

This week we publish a new long-form report, Cloud e-discovery: litigation comes down to earth – download an executive summary here.

In cloud e-discovery we see two major market shifts: corporations in-sourcing e-discovery to lower costs, while outsourcing IT infrastructure and services around it through hosting.  Still in early adoption, it is a leap of faith on some level, and carries both risks and benefits.  While most users in our 2010 e-discovery survey were bringing the e-discovery process in-house, only 16% were using cloud to do it, for a variety of reasons including security, data loss, regulatory concerns, and ease of retrieval.

But consider that hosted e-discovery has actually been around for over 20 years. What’s more, while some enterprises are resisting the cloud, their law firms, service providers, and other outsourcers entrusted with their data are not.

Witness this month’s 2010 Am Law tech survey – 80% of law firms are using hosted technology, 60% of those for e-discovery.  In fact, e-discovery tops all hosted software usage, far surpassing HR (21%), spam filter/email (21%), storage (6%) or document management (5%).  And while 79% report a positive experience, 30% said the savings were not what they expected.  Limited customization, diminished data control and security were even greater concerns.

And what of the bigger-picture risks?  Cloud topped the agenda last month at the Masters Conference as well: the growth of public and private cloud data from mobile use and social media, potential regulatory pitfalls, the benefits and risks of hosted e-discovery, and growing cross-border issues.  No blue-sky thinking here, just hard truths on the cloud from those on the front lines.

From e-discovery lawyers and consultants:

• “[Public] cloud providers can’t meet the needs [of e-discovery] today.”
• “Your data, your problem.”
• “Data privacy in the EU is like free speech or freedom of religion in the US. . . they will give up the cloud before they give this up.”

From Microsoft General Counsel, speaking on cloud regulation:

• “Things will move quickly, and if something bad happens, things will move faster still.”

From an enterprise buyer on procurement:

• “It will take 19 months to work out e-discovery issues once you start talking about it.”
• “Every dollar they save on cloud will be three dollars in legal.”
• “I hate when people say ‘it’s not gonna stop – it’s already there.’ It makes customers think there is no choice but to comply.  But maybe ‘cloud’ will go away?”

And for the last word, a characteristically common-sense admonition from UK expert Chris Dale (speaking on ECA):

• “The best technology lies between your ears.”

So, how to navigate it all?  For a succinct analysis of the cloud e-discovery market, our report is available to 451 CloudScape or Information Management subscribers, or get an executive summary here.  It offers a market overview, benefits and risks of cloud e-discovery, adoption trends and inhibitors, market drivers, current vendor and service-provider offerings, and the future direction of the market, particularly for enterprise customers.

Also note a complementary report, Cloud archiving: a new model for enterprise data retention, by Simon Robinson and Kathleen Reidy.  They estimate the market will generate around $193m in revenues in 2010, growing at a CAGR of 36% to reach $664m by 2014.  This report covers growth drivers, the competitive landscape and the outlook for consolidation, featuring detailed vendor profiles and end-user case studies.

The cloud archiving market will generate around $193m in revenues in 2010, growing at a CAGR of 36% to reach $664m by 2014.

This is a key finding from a new 451 report published this week, which offers an in-depth analysis of the growing opportunity around how the cloud is being utilized to meet enterprise data retention requirements.

As well as sizing the market, the 50-page report – Cloud Archiving; A New Model for Enterprise Data Retention – details market evolution, adoption drivers and benefits, plus potential drawbacks and risks.

These issues are examined in more detail via five case studies offering real world experiences of organizations that have embraced the cloud for archiving purposes. The report also offers a comprehensive overview of the key players from a supplier perspective, with detailed profiles of cloud archive service providers, with discussion of related enabling technologies that will act as a catalyst for adoption, as well as expected future market developments.

Profiled suppliers include:

• Autonomy
• Dell
• Global Relay
• Google
• i365
• Iron Mountain
• LiveOffice
• Microsoft
• Mimecast
• Nirvanix
• Proofpoint
• SMARSH
• Sonian
• Zetta

Why a dedicated report on archiving in the cloud, you may ask? It’s a fair question, and one that we encountered internally, since archiving aging data is hardly the most dynamic-sounding application for the cloud.

However, we believe cloud archiving is an important market for a couple of reasons.  First, archiving is a relatively low-risk way of leveraging cloud economics for data storage and retention, and is less affected by the performance/latency limitation that have stymied enterprise adoption of other cloud-storage applications, such as online backup. For this reason, the market is already big enough in revenue terms to sustain a good number of suppliers; a broad spectrum that spans from Internet/IT giants to tiny, VC-backed startups. It is also set to experience continued healthy growth in the coming years as adoption extends from niche, highly regulated markets (such as financial services) to more mainstream organizations. This will pull additional suppliers – including some large players — into the market through a combination of organic development and acquisition.

Second, archiving is establishing itself as a crucial ‘gateway’ application for the cloud that could encourage organizations to embrace the cloud for other IT processes. Though it is still clearly early days, innovative suppliers are looking at ways in which data stored in an archive can be leveraged in other valuable ways.

All of these issues, and more, are examined in much more detail in the report, which is available to CloudScape subscribers here and Information Management subscribers here. An executive summary and table of contents (PDF) can be found here.

Finally, the report should act as an excellent primer for those interested in knowing more about how the cloud can be leveraged to help support ediscovery processes; this will be covered in much more detail in another report to be published soon by Katey Wood.

Katey and I are doing a few webinars at the moment and I’m also speaking at a conference this week, so I just wanted to round them all up here:

One webinar is already in the bag, which Katey did with Digital Reef & legal service provider Precise-Law, entitled ‘The challenges of a  reactive vs. proactive EDRM in the Enterprise.’ A replay is available here.

I’m speaking at Search Solutions 2010 this week  on Oct 21. It’s a one-day event organized by the British Computer Society, which I attended last year as a non-speaker and it was very good, so I hope to be able to contribute to maintaining that high standard! I’m speaking at 11.45 am on ‘The trends shaping the future of enterprise search 2010-2013’ and then I’m participating on a panel at the end of the day on what search will look like in 2015. As I’m already making predictions through 2013, I’m three-fifths of the way there! Oct 21 is also the day of Autonomy’s Q3 results call so the place should be full of lively discussion regarding that.

Come November I’m doing a couple more webinars:

On Nov 11 I’m participating on one with Zylab, the focus of which will be litigation-readiness, moving beyond just eDiscovery to insuring organizations have their information in a state such that it can be easily searched, accessed, locked down, deleted or produced to an opposing party.

Also in November I’ll be participating in a webinar with Attensity Group, which will be focused on social media and the application of text analytics to that space. Date TBC and links to follow, most likely on my Twitter feed.