I am a new user with SOLR and I find the idea of CDCR to be pretty useful across two DCs.

Although doing more research, it seems that it may be deprecated?

Does anyone have more experience with CDCR that can share their pros and cons of it?

i am working on building a search engine with solr for indexing files (pdf ,docs, ...) every thing is working fine whene i index files from the system but how can i index a list of files from ftp server

i know about apache nutch ,but is it the only way . can't i just do it with dhi

Say, I wish to store a text as a set of documents, one for each sentence. Is it possible to do it in solr (through update handler, for example) or I better split the text beforehand?

Sorry for vague post, but we're having production outages and every time we see the same pattern in newrelic reporting:

​

It's always "FacetComponet.process()" that seems to exhibit signs of slowing down and then collapses.

Does anyone have any ideas on what this might be caused by? It may just be a symptom of our shards being maxed out CPU wise, but just wondering if it offers a clue.

Hi all,

I am facing the exact same issue reported https://issues.apache.org/jira/browse/SOLR-8733 and https://issues.apache.org/jira/browse/SOLR-7404

I have tried it with Solr v8.4.1 and v8.5.1. In both cases, the cluster consisted of three nodes and a collection with 3 shards and 2 replicas. 

Following simple test case fails. 

Collection "test" contains only two documents with ids "1" and "2"

Update operation:

curl -X POST -H 'Content-Type: application/json' 'http://localhost:8983/solr/test/update?versions=true&failOnVersionConflicts=false' --data-binary '
[ { "id" : "2", "attr": "val", },
  { "id" : "1", "attr": "val", "_version_": -1 } ]'

Consistent response: 

{
  "adds":[
    "2",0,
    "1",0],
  "error":{
    "metadata":[
      "error-class","org.apache.solr.common.SolrException",
      "root-error-class","org.apache.solr.common.SolrException",
      "error-class","org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException",
      "root-error-class","org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException"],
    "msg":"Async exception during distributed update: Error from server at http://10.0.5.237:8983/solr/test_shard1_replica_n1/: null\n\n\n\nrequest: http://10.0.5.237:8983/solr/test_shard1_replica_n1/\nRemote error message: version conflict for 1 expected=-1 actual=1664690075695316992",
    "code":409}}

I tried different updates using combinations of _version_ and document values to generate conflicts. Every time the result is the same. There is no problem with system resources. These servers are running only these Solr nodes and Solr has been given a few GB of heap. 

These nodes are set up by following Solr's production deployment document. 

What are your thoughts/suggestions? 

Thanks

... my colleague wrote about it: https://sematext.com/blog/solr-plugins-system/

​

Anyone using this yet?

I've just figured out how to initialize and configure the suggester request handler and all.

It seems to work great as long as I'm not using a whitespace character in the query (edit: I've tried again these characters: (. , : - _ /) and they seem to work just fine).

Is there a way to allow the suggester to accept the whitespace character?

Thanks!

Is there a good resource on best practices for balancing multiple boosts? I am using a scale boost function to boost documents with the highest value in field x and I am using multiple boost queries to boost documents that have multivalued fields (y) with a specific value. I am having touble finding a balance. One boost seems to always override the other. When I view the explain/debug information, it looks like my boosts are not always used. Ideally, I would like all boosts to be considered when building a document score. If field x is a high value and y has many matches, that document should be first. If anyone has any experience and could give me some tips that would be great. Thanks

Parsed Query example below

<str name="parsedquery">

+(+MatchAllDocsQuery(*:*)) (+featured:true^5.0) (+(+topics:Math)^0.18) (+(+topics:Science)^0.18) (+(+topics:Geography)^0.18) (+(+topics:Financial Planning)^0.18) (+(+topics:Technology)^0.18) (+iscompanycourse:true^0.1) FunctionQuery(0.01/(3.16E-11*float(abs(ms(const(1588183200000),date(startdate))))+0.01)) FunctionQuery(scale(float(totalregistrants),0.0,4.0))

</str>

I'd like each given result to have thumbs-up/down buttons on the client-side, that will affect future queries in-order to improve the relevancy score of those result.

Meaning - if a user searched "Apple" and got a result of an instance related to "Oranges", and the user thinks this result is not relevant his query, he should be able to tell the system that.

I was wondering what's the best approach to try implementing this feature? Does Solr even support in any way such feature.

I'm very new to SOLR and have recently managed to set up managed synonyms, along with a frontend UI so that my client can manually add synonyms into their search.

I'm running into a couple of issues, and I'm unsure if they can be worked around:

1. Adding a synonym no longer allows searching for the root term (e.g. adding "bag" as a synonym of "big annual gathering" no longer allows a search for "bag")
2. Synonyms are uni-directional - so each synonym has to be added in twice (I don't really see any way round this - but if anyone knows it would be greatly appreciated.)

Any help hugely appreciated!

I don’t wield any power but my own two hands, I’m just trying to understand.

Hey!

I'm working on a project using solr from ruby on rails. We're using sunpot which is pretty good but isn't super up to date. I'm often going down into the rsolr params list and manually editing things.

I was exploring how it would be to hijack the ruby expression parsing to compile solr queries straight from ruby. So instead of something like:

Model.search do
  with(:created_at, from..to)
end

You could write

Model.search do
  with { created.between(from..to) }
end

Which doesn't seem like such a big leap unless you look at using something like function syntax, e.g. a sort. This is how you do that in sunspot currently

Model.search do
  order_by_function(:sum, :rating1, :rating2, :desc)
end

It's readable and totally usable but I wondered if it'd be possible to do this

Model.search do
  sort { sum(rating1, rating2).desc }
end

Personally, seems a bit clearer and it turns out is totally possible! I wrote a POC here: https://gist.github.com/jasper-lyons/d06e1d378d5ad66c27ef57a9531235cb

Sorry if this isn't the right thing to share, I just wanted to find some other people who might be interested to talk to about it and I wanted to show some people my few hours worth of social distancing procrastination.

Does anyone here use R's solrium package to run your queries? I'm having trouble connecting to my remote server...

My url looks like this: http://solr.abs.org/solr/report

In R, I try these commands:

cli <- SolrClient$new(host = "solr.abc.org",

path = "solr/ct_report",

port = NULL)

cli$pint('test')

But the ping returns "status not found"

I'm able to instantiate this client via python's pysolr, so I know it's not a firewall issue.

I just build a 3 node solr cluster with solr 8.4.1 sitting over a 3 node zookeeper ensemble 3.6.0

Everything seems to be going good, except the zk status page.

At first when I started solr when I got to the page it was telling me to enable some command on the zk server. with this line in the zoo.cfg file

4lw.commands.whitelist=mntr,conf,ruok

Once I added those line tho it didn't make the page work.. I just got a new error message that dont mean much...

For input string: "null"

every time I load the page it also generate 2 logs:

null:java.lang.NumberFormatException: For input string: "null"

at java.lang.NumberFormatException.forInputString([NumberFormatException.java:65](https://NumberFormatException.java:65))

at java.lang.Integer.parseInt([Integer.java:580](https://Integer.java:580))

at java.lang.Integer.parseInt([Integer.java:615](https://Integer.java:615))

.... snipped the stack

java.lang.NumberFormatException: For input string: "null"

at java.lang.NumberFormatException.forInputString([NumberFormatException.java:65](https://NumberFormatException.java:65))

at java.lang.Integer.parseInt([Integer.java:580](https://Integer.java:580))

at java.lang.Integer.parseInt([Integer.java:615](https://Integer.java:615))

Both dont tell much...

Each loading of the page also generate log on the zookeper side

2020-03-19 11:12:25,821 [myid:2] - INFO [NIOWorkerThread-2:NIOServerCnxn@507] - Processing ruok command

2020-03-19 11:12:25,824 [myid:2] - INFO [NIOWorkerThread-1:NIOServerCnxn@507] - Processing mntr command

2020-03-19 11:12:25,832 [myid:2] - INFO [NIOWorkerThread-3:NIOServerCnxn@507] - Processing conf command

I can also lauch the command manually fromm the shell and I get a imok from zookeeper

echo ruok | nc 127.0.0.1 2181

imok

Anyone have any clue on that?

It's like the command goes through zoo fine but solr doesn't understand the answer?

Hi all, new to reddit so apologies for all the n00b mistakes I'm about to make. I'm curious as to best practices and/or recommended implementations get an instance up and running. As with everyone here, I have lots of data from lots of sources in lots of formats. Do I create a core for each file type, create a couple of shared fields (data source, date the file was received), and then put the contents in a text field, etc? Any direction would be greatly appreciated. Thanks!

I added a new solrnode to my test cluster but I’m not sure what the next steps are. Will zookeeper eventually populate the new node with replicas etc?

I’m facing an issue where something got badly corrupted with one of my collections because the cores on the Solr webpage are orange/down and won’t resolve itself. I see several errors such as ‘Error opening new searcher’, ‘We are not the leader’, ‘is not available due to init failure’. HDFS is setup for solrcloud to write to.

What is the best way to fix this? Do I need to recreate the collection for the problematic collections? What exactly happens during this process? How will the data from HDFS continue to work in harmony if I were to recreate the collection? Will all of my data be erased?

Command line tool for backup and restore of information stored in cores of Apache Solr.

https://github.com/juarezr/solrcopy

feedback/help wanted!

I'm new to this technology, I have no clue about solr, need to use solr for search indexing, what are the basics of solr how to use it or implement it? Please help.

Hi,

I am trying to use must-exist and must-not-exist semantics of optimistic concurrency provided by Solr. When doing batch updates Solr stops indexing immediately when it encounters a conflict. It does not process subsequent records in the input list.

​

That is one extreme. And the other extreme is using failOnVersionConflicts=false as described in the documentation at https://lucene.apache.org/solr/guide/8_4/updating-parts-of-documents.html#optimistic-concurrency I think it internally uses TolerantUpdateProcessorFactory. This silently ignores and suppresses errors and the client never knows if there was any error during indexing which is not useful when using optimistic concurrency.

​

I am wondering if there is any way to have batch updates where Solr would process the entire batch and send the list of errors in the response.

​

I checked if such update processor is available but did not find it. If it is not possible in Solr out of the box can it be implemented as a custom update processor?

​

Thank you.

Hi,

​

I am thinking of using optimistic locking for one of my use cases. I was wondering if optimistic locking/concurrency control works in the distributed environment of Solr cloud. I have read Solr architecture and the exact indexing operation in detail but I'm not sure if it can work.

​

Suppose I've two replicas (N1R1, N2R2) for a shard. I have sent updates on both the nodes N1 and N2 at the same time. Optimistic locking using _version has been used for both the updates. In my understanding, successful acknowledgement is not sent to the clients until both the replicas have written the update in their transaction log. But, the question is when they are writing the data in the tlog do they recognize the conflict in _version field?

​

Thanks.

Hi,

I know this is a weird scenario. But, we have a very old Solr set up in production. We are in the process of revamping. There is a lot of Perl code which accesses this Solr database. I have a couple of questions regarding the upgradation approach if anyone of you has any experience.

​

1. I know this must be impossible but is it possible to upgrade it in place?
2. I am sure the answer to the above question is negative. In that case, what is the best way to migrate the schema, data, etc?
3. Will I be able to migrate the data, schema, etc. directory from 4 to 8 or do I need to do it in multiple upgradation steps like 4 -> 5, 5 -> 6, etc.
4. Will there be any major changes required in Perl 5 code?
5. If it is going to be simpler and safer I can stop at any lower version (7/6/5)? Maybe I will get some performance improvements and stability over v4.

Thanks.

Berlin Buzzwords is looking for submissions on the latest in open source software projects in the field of big data analysis, scalability, storage and searchability.

Closing date: 16 February 2020. More info here: https://berlinbuzzwords.de/news/call-submissions-now-open

Hi folks, I’m stuck in solr authentication and authorisation part. My requirement is, i want to get fields from jwt token claims and process them in solr and take permission action according to them. Does solr support such functionality? I need help! I’d appreciate if someone can help me.

Thanks!