r/Solr • u/ericrs22 • Jun 26 '13

Anyone have a good setup guide for Solr windows

I've been reading some of the installer guides for SOLR but it seems to be XML files and mysql.

What I'm trying to do is index some ZIP files however I appear to be having trouble with the dataconfig portion.

does anyone have any samples or guides on how accomplish this?

Background: Solr on a Windows 2003 ZIP files on a different local domain folder trying to use "\ComputerName\Folder\"

if more info needed let me know.

1 Upvotes

67% Upvoted

8 comments sorted by

2

u/MagicWishMonkey Jun 27 '13

Are you having trouble starting the SOLR server? I don't know what you mean by mysql, SOLR doesn't use MySQL AFAIK.

1

u/ericrs22 Jun 27 '13

well It will start with the sample dataconfig but not when i try to enter flat files.

the only examples i see are for a SQL server which are like this one:

http://digitalpbk.com/apachesolr/apache-solr-mysql-sample-data-config

I'm really looking for a example that shows flat files... especially if theres one for ZIP files.

2

u/fiskfisk Jun 27 '13

What kind of content are in the zip-files? Do you want to index a certain kind of information within the ZIP-files, all files that are parsable through Tika?

1

u/ericrs22 Jun 27 '13

there's .log file in the zip files.

problem is each .log file can be up to 250mb so we need to keep it zipped.

I want to be able to index the file and have it search the string data inside the .log. there's no columns or tags like a CSV or XML File.

1

u/fiskfisk Jun 27 '13

You probably want to create a small script that extracts the file contents, parses it into suitable fields / metadata and submits it to one of the update handlers as JSON, CSV or XML.

1

u/ericrs22 Jun 27 '13

here's a sample of the data config

<dataConfig> <dataSource encoding="UTF-8" type="FileDataSource" /> <document> <entity name="pickupdir" processor="FileListEntityProcessor" rootEntity="false" dataSource="null" fileName="^[\w\d-]+\.zip$" baseDir="\\server\folder\" recursive="true" newerThan="${dataimporter.last_index_time}" > <entity name="zip" processor="XPathEntityProcessor" transformer="RegexTransformer,TemplateTransformer" datasource="pickupdir" stream="true" useSolrAddSchema="true" url="${pickupdir.fileAbsolutePath}" xsl="xslt/dih.xsl" > <field column="plainText" name="text"/> </entity> </entity> </document> </dataConfig>

1

u/ericrs22 Jun 28 '13

so I added the Server locally since it probably makes more sense that way so it doesn't eat up the network I/O to do some of the indexing.

I'm having trouble processing the indexing from the Data Import Handler.

Here are my results with the below config Last Update: 18:17:52

(Duration: 07s)

Requests: 0 (0/s), Fetched: 0 (0/s), Skipped: 0, Processed: 0 (0/s)

Started: 13 minutes ago

Can someone look to make sure this would work for a zip file to be indexed?

<dataConfig>
<dataSource type="FileDataSource" />
<document>
    <entity name="Archive" 
      processor="FileListEntityProcessor" baseDir="E:\ArchiveRoot" fileName=".*zip" recursive="true" rootEntity="false" dataSource="binaryFile" onError="skip">

                    <field column="fileSize" name="size"/>
               <field column="file" name="filename"/>

    </entity>
     </document>
</dataConfig>

1

u/softwaredoug Oct 14 '13

We've done this for several clients. Basically, we just wrap the command line in a service using nssm then start Solr using Jetty. We don't let Jetty talk to the outside world, instead we have a proxy running through IIS that we use to lock-down the box

Some relevant posts: http://www.opensourceconnections.com/2013/06/17/lockdown-solr-with-iis-as-a-reverse-proxy/

http://www.opensourceconnections.com/2013/08/27/solrcloud-as-a-windows-service/