Issuu on Google+

Practical Search in the Cloud

(sneak preview) Martijn Koster, Lucid Imagination martijn.koster@lucidimagination.com

Wednesday, 25 May 2011


My Background  Martijn Koster  Sr. Software Engineer at Lucid Imagination • Search in early Web – WebCrawler • Customer web services – Excite Mail • Proxies for Mobile – Danger

2

Wednesday, 25 May 2011


What I Will Cover     

A Solr based Cloud Service The need How Demo Wrap up

3

Wednesday, 25 May 2011


Why? – Getting Started is Hard  Many new concepts  Complex software configuration  System Administration overhead

4

Wednesday, 25 May 2011


Search as a Service  A few clicks to get started  Full Power of LucidWorks and Solr  Offload operations  Expertise, tools and processes are transferable to your own deployments.

5

Wednesday, 25 May 2011


Usage Model Users Web UI LWELucidworks JVM LWE JVM Solr

User Code Direct Access

6

Wednesday, 25 May 2011


Architecture 3rd party billing Users

Web UI (Rails)

Heroku Provisioner (Ruby)

Heroku User Code

Direct Access (Proxy with URL rewriting)

Control (Chef)

Provisioning Service (Rails)

LWE JVM LWE JVM LWE JVM

SQL (RDS)

Monitoring (Zabbix) 7

Wednesday, 25 May 2011


Deployment Automation  AMI  Chef  Custom

8

Wednesday, 25 May 2011


Scaling  Heroku  Direct Access  Search Servers  Provisioning/DB  Monitoring and Control

9

Wednesday, 25 May 2011


Direct Access Example  The Cloudy Direct Access Solr interface • Simple submission • Simple search First, copy SOLR access URL:

SOLR="https://access.cloudy.io/…/solr"

11

Wednesday, 25 May 2011


Example - Add $ cat > /tmp/doc.xml <<EOM <add><doc> <field name="id">testdoc_LR2011</field> <field name="text_en">LR rocks!</field> </doc></add> EOM curl $SOLR/my_news/update?commit=true \ -H "Content-Type: text/xml" \ --data-binary @/tmp/doc.xml

12

Wednesday, 25 May 2011


Example - Search curl $SOLR'/my_news/lucid? q=news&role=DEFAULT&wt=json' curl $SOLR'/my_news/select? wt=json&q=news&start=0&rows=10'

13

Wednesday, 25 May 2011


Example – Lucidworks API • The Cloudy Direct Access Lucid API interface API="http://access.demo.cloudy.io/…/api”

curl $API/collections [ {"name":"lwelogs","instance_dir":"lwelogs” {"name":"collection1","instance_dir":"collection1_0"}, {"name":"my_news","instance_dir":"my_news_1"} ]

14

Wednesday, 25 May 2011


Live Demo  Sam Pierson, Director of Product Development  The Web User Interface • Administration UI improvements • Log indexing and graphing

 API access • Ruby bindings for the LucidWorks REST API • http://rubygems.org/gems/lucid_works

10

Wednesday, 25 May 2011


Lessons  Cloud technologies speed up development  Automate everything  Avoid lock-in  Dogfooding

16

Wednesday, 25 May 2011


Wrap Up  Search as a Service  Built in the Cloud, with open source  Full Lucidworks Enterprise and Solr  Powered by Lucid Imagination

17

Wednesday, 25 May 2011


When  Early Access Program: July • Interested? cloud@lucidimagination.com

18

Wednesday, 25 May 2011


Contact

 Martijn Koster • martijn.koster@lucidimagination.com • http://www.lucidimagination.com/ • twitter.com/makuk66

19

Wednesday, 25 May 2011


Practical Search in the Cloud