Get only new items from Firebase

1_query_timestamp.js

// assumes you add a timestamp field to each record (see Firebase.ServerValue.TIMESTAMP)
// pros: fast and done server-side (less bandwidth, faster response), simple
// cons: a few bytes on each record for the timestamp
var ref = new Firebase(...);
ref.orderByChild('timestamp').startAt(Date.now()).on('child_added', function(snapshot) {
  console.log('new record', snap.key());
});

2_once_value.js

// pros: works?
// cons: fetches entire record set, requires discarding the first record
// credits: http://stackoverflow.com/questions/12850789/is-there-a-way-to-know-in-what-context-child-added-is-called-particularly-page/12851236#12851236
var ref = new Firebase(...);
ref.once("value", function(snap) {
  //TODO: display initial state...
  // Object.keys not supported in IE 8, but has a polyfill: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/keys
  var keys = Object.keys(snap.val()||{});
  var lastIdInSnapshot = keys[keys.length-1]

  ref.orderByKey().startAt(lastIdInSnapshot).on("child_added", function(newMessSnapshot) {
    if( snap.key() === lastIdInSnapshot ) { return; }
    console.log('new record', snap.key());
  }
}

3_limit_endat.js

// pros: does not fetch entire record set
// cons: triggers child_removed events when last item changes, grabs the last record which needs to be ignored
var first = true;
var ref = new Firebase(...);
ref.limitToLast(1).on("child_added", function(snap) {
   if( first ) {
       first = false; 
   } 
   else {
       console.log('new record', snap.key());
   }
});

@katowulf, any thoughts on why 3_limit_endat.js doesn't work well on larger datasets? It triggers as expected on a node with 10 records, but does not trigger on a node with 400,000 records.

I think I like the 1_query_timestamp.js method best. But is there a way to not use the client side date but the server side date at startAt?

@ngerritsen firebase.database.ServerValue.TIMESTAMP

Edit: this does not work on startAt
http://stackoverflow.com/q/39437845/1325646

The problem with 1_query_timestamp.js is that client time may be out of sync with server time. As mentioned firebase.database.ServerValue.TIMESTAMP cannot be used with startAt.

There is an approach to get the servertime, see: http://stackoverflow.com/questions/23128027/retrieve-firebase-server-time-without-setting-it-first.

Kato, on 3 you remark 'triggers child_removed events when last item changes'

can you explain that ? what does child_removed has to do with it ?

also isn't 3 missing orderByKey() ?

@kofifus I think he means that when you first run the code, it returns the last already-existing item in the database. Whereas, you actually only want to know about new items that will be added in the future. Hence, it requires that if statement in the handler to ignore the first entry.

Thank you ! That worked for me.

It works! Awesome! Respect 🙇‍♂️

how can i use timestamp with python? I am going to save image from python to database and want to retrieve the latest image uploaded from database

Thanks for your solution ! It's work for me
🥇

dont forget to add an index for the timestamp to make the filtration quicker https://www.firebase.com/docs/security/guide/indexing-data.html

This is also one solution,
ref.startAt(null,ref.push().key).on('child-added,...'
works only when keys are generated by push()

This is also one solution,
ref.startAt(null,ref.push().key).on('child-added,...'
works only when keys are generated by push()

This is interesting, do you know why this works? Are keys generated in order by RTDB?

On 2), ref.orderByKey().startAt(lastIdInSnapshot).on("child_added"..., I found out that instead of throwing away the first value, you can add a '-' to the lastIdSnapshot. It's just an character with a low ASCII value, so any push'ed item will go after the previous one + '-'.

This is interesting, do you know why this works?

if we were to use ref.push().key to put a new entry on ref, it would be stored as the last child.So, using second parameter of startAt function, we retrive childs with key greater than or equal to ref.push().key
https://firebase.google.com/docs/reference/js/firebase.database.Query#startat

Are keys generated in order by RTDB?

yes, https://firebase.google.com/docs/database/admin/save-data#push-vs-transaction