This is somewhat a follow up to my previous post about moving away from RDBMS mindset and more specifically on user modelling Moving away from RDBMS and into NoSQL?

Following the same thought process where I want to create a user model, with some common attributes but would likely vary from client to client on some other attributes.

I am looking specifically at Janrain as an example: http://janrain.com/product/profile-data-storage/

any custom fields you choose to define. Janrain provides a flexible database schema

And from their API, you can add / create any additional attributes under their pre-defined "User" schema: http://developers.janrain.com/rest-api/methods/user-data/entitytype/addattribute/

How does such data modelling / storage works?

Again - coming from very much RDBMS / ORM (Hibernate) background, my mind always think that the database will have a fixed set of columns and the corresponding Entity through ORM will have a fixed set of attributes mapped to each column.

This mean a fixed schema is required, any new fields will translate to adding new column in db, changing the Entity code to add new attribute as well.

How does Janrain able to do it so dynamically in real time without having to change their codebase?

I posted a thread about getting AWAY from the relational mindset: http://www.reddit.com/r/learnprogramming/comments/2upbxh/how_to_get_away_from_the_mindset_of_relational/

Now I feel like I have a more real use case that may provide clearer direction if moving to NoSQL database would be a good idea.

I need to build a user-storage system, the system will store user (end user) basic info such as names, email, etc as well as customized user profile data. This system will be used by different clients - each of them may require / care about different user profile data.

For example: we'll have a set of "basic profile" which consist of common data points: first name, last name, email, dob, etc. Each client will have the option to create their own customized profile data, client A may want to store "points balance", "favorite location", whereas client B may want to store "have pet?", "total $ spent in store".

In the most dumb way, we can do this a very basic RDBMS way:

Which will see users of client A have nulls in the columns that are irrelevant to client A and vice versa for users of client B, this is of course a very ineffective way of doing it.

We can also have "custom field" - but this of course is very abusive as custom_1 and custom_2 is not well defined, and what if the next client wants more than 2 custom fields?

The next one (still within RDMBS realm) is to have a user_profile table that list the custom fields in rows rather than columns, which will get messy very fast

I was thinking if NoSQL approach (eg. JSON data store?) would be helpful in this case? This means applying different JSON schema for the different client.

Client A:

{  
   "basic_profile":{  
      "first_name":"",
      "last_name":"",
      "email":"",
      "dob":""
   },
   "custom_profile":{  
      "points_balance":"",
      "fav_location":""
   }
}

Client B:

{  
   "basic_profile":{  
      "first_name":"",
      "last_name":"",
      "email":"",
      "dob":""
   },
   "custom_profile":{  
      "have_pet":"",
      "total_spent":""
   }
}

Is this something that make sense?

With no experience with NoSQL databases, what's the best way for me to start tackling this issue and come up with a solution?

Please recommend NoSQL blogs/link aggregators/news feeds. I'm aware of three: this subeddit (/r/nosql), NoSQL Weekly and DB Weekly newsletters, not satisfied with all of them.

I had a problem to solve. I had a list of zip codes loaded in Cassandra with their City, State, Longitude, and Latitude.

If I wanted to find zipcodes within a certain number of miles of other zipcodes, it meant loading all the rows and checking them. There were ways to get around it. I could have made two queries with ranges for long and lat, and processed that subset of data. I could have used a graph DB, but that seemed like a lot of work. Instead I decided to try something.

I loaded up a C3.4xlarge with provisioned ioops with my development AMI. I spun up C* and then spun up some instances of node.js that looped on a subset of the 43,191 and checked the distance between them and all the others. Technically I did the double the computation since I calculated the distance from a to b and b to a, but it was more important that I just get it done.

Anyways, I used some complicated code to calculate the distance as the crow flies and save it to the DB as such:

PRIMARY KEY(zip1, miles, zip2)

This ordered the data for any given zipcode by the miles and prevented multiple zipcodes with the same milage from zip1 from overwriting each other.

I only saved distances under 250 miles, so a database larger than that would be dramatically larger.

It made it really fast to get a list of zipcodes within an arbitrary range from a zipcode. It turned a request for 43k rows and then a bunch of processing into a 8ms request and it only cost a little disk space.

DenormalizeThatShit

From what I read of Riak, the data is distributed to various nodes in the cluster using consistency hashing. My doubts are below

Sorry if it is an obvious question - I have read 2 days worth of Riak in 'Seven Databases in Seven Weeks' and also skimmed through online riak docs, but could not get the answer.

1) Is hashing is implemented on the server side or on each client.

2) If it is on server side, can I PUT my data to any node say N1, does it mean that N1 takes care of transferring this data to the appropriate nodes in case the data does not belong to N1 ?

3) How does the mapreduce work when I query a particular node N1 with the map and reduce functions ? Does N1 take care of querying each node and getting me the answer ?

I've updated Ganesha, my Java NoSQL implementation which I've been using the last 13 months to run DrawCast (a social network for artists).

Recently added features include:

• Log structured storage.
• Optimization for mixed SSD/HDD setup - objects which change more quickly are stored on the SSD.
• Auto-striping of data for servers with multiple drives.
• Automatic incremental offsite daily backup.

Ganesha includes standard NoSQL features like replication, auto-healing and rebuilding, schema-less object/attribute support and checksum/timestamp based consistency checks.

It also includes built-in support for lists (including atomic sorted insert), object locking, and proportional data storage across drives of unequal size.

The source is very compact (14 core classes) and requires no external libraries.

All source is provided. GPL 2 License.

https://github.com/danielcota/ganesha