The current description of this repo read as follows:
"Firebase Utilities: A Firebase wrapper that joins several paths either as a union or intersection of the merged data sets."

But by what I've read in the updated docs, and by what I've tried to fiddle on my own, the union aspect has been taken out of this tool, right? As the normalized collection tries to merge different properties of different paths, but would not work in a path like this one:

/:activeTasks/:userId/[:taskId, :taskId]
/:completedTasks/:userId/[:taskId, :taskId]

In this case I would only need the joined path of both nodes.
Is this functionality really taken out of the toolset (and therefore the description outdated) or am I missing something?

Thanks

Hey kato.
I asked you a month ago about the compatibility with the new firebase and you said you would eventually get into it. Not trying to press the same button twice but do you have any idea of when can we expect that to be?

I'm using your module on a production app that will be shipped on April 2015.
It's working wonders even with the new firebase. Only problem so far is the huge number of warnings.
Are you planning on working on it again in this next 4 months or is this not really in your plans right now?

I would gladly help if it's just a matter of suppressing the warnings by changing the name() to key() methods. I'm not very confident in doing anything beyond that though...

Cheers

At some point, add some rudimentary reversion controls that will attempt to write in an atomic manner and revert changes on a failure.

The cdn link appears to be broken

I'm trying to follow the paginate demo and have run into an issue where the console is returning this error:

ReferenceError: $scrollArray is not defined

My HTML and JS looks like this:

services.html

<h3>Items loaded: {{scrollItems.length}}</h3>

app.js

(function() {
  'use strict';

  var outcomesServices = require('./services/services.js');

  angular.module('outcomesApp', [
    'firebase',
    'outcomes.services'
  ])

  .constant('FBURL', 'https://newer-outcomes.firebaseio.com/');
})();

services.js

angular.module('outcomes.services', ['ngRoute'])

.controller('ServiceListController', ['$scope', 'FBURL', '$firebaseArray',
  function($scope, FBURL, $firebaseArray) {
    var ref = new Firebase(FBURL);
    var servicesRef = ref.child('services');
    $scope.scrollItems = $scrollArray(servicesRef, 'number');
}])

.factory('$scrollArray', function($firebaseArray) {
  return function(ref, field) {
    var scrollRef = new Firebase.util.Scroll(ref, field);
    var list = $firebaseArray(scrollRef);
    list.scroll = scrollRef.scroll;

    return list;
    console.log(list);
  }
})

Any idea what I'm doing wrong. I've never really used the .factory() provider before, so not sure if I'm doing this correctly.

Any help on this issue is appreciated. Thanks in advance!

hey man,

are you planning on adapting your lib to the new firebase methods like .key and the .limitToFirst ?
this thing is working beautifully but this warning are making impossible to debug my app.

cheers and thanks for this awesome tool!

Hi, I'm trying to load firebase-util in a NodeJS program but when I require the module the result is an empty object.

This is what I do:

var FirebaseUtil = require('firebase-util');

FirebaseUtil.NormalizedCollection('...') << NormalizedCollection is undefined
FirebaseUtil.escapeEmail('...') << escapeEmail is undefined

It fails even if I do this:

var FirebaseUtil = require('./node_modules/firebase-util/dist/firebase-util.min.js');

FirebaseUtil.NormalizedCollection('...') << NormalizedCollection is undefined

Am I doing something wrong or it is a bug?

Trying to order my query by a timestamp:

 scrollRef.orderByChild('timestamp')

orderByChild is not supported for Paginate and Scroll references. Try calling it on the original reference used to create the instance instead.

rootRef.orderByChild('timestamp')

Query.orderByChild: You can't combine multiple orderBy calls.

Thanks for the great work on the latest release.

I'm looking to use the Scroll class on a collection of comments, where they each have a timestamped createdAt field. The items need to be ordered with the most recent at the top. I can pass createdAt to the Scroll class, but it uses orderByChild which orders them with the oldest first.

Scroll seems to only lets you pass a raw firebase reference to it, not a Query, so I can't use limitToLast() to reverse the order that the items are loaded in.

When using limitToLast on the scrollRef, it throws:

limitToLast is not supported for Paginate and Scroll references. Try calling it on the original reference used to create the instance instead.

But when using limitToLast on the original reference as indicated, it throws:

First argument to Firebase.util.Scroll must be a valid Firebase ref

Reversing the collection on the client doesn't seem like a possibility because Scroll is loading comments in via "pages" from the opposite end of the collection. Hopefully that makes sense...

I understand Scroll is using Queries internally, so passing it a query would break it since chaining Queries isn't possible, but is there an optimal strategy for this?

Not sure if this is an issue or if I don't know how to configure firebase-util correctly, but I am having problems accessing records in the following cases (when I apply firebase security rules)

List of posts from a given user:
/users/{userId}/posts/{postId}/true

All posts with detailed information (and a list of users who have access to them)
/posts/{postId}/users/{userId}true

I have configured my firebase access rules to:

• Allow read access to the posts list for authenticated user
• Allow read access ONLY to the posts information which have the user in its users list

When I use a firebase-util intersection , I get an empty result because the user does not have read access to the whole /posts ref (if open full read access it works).

I guess that if firebase-util intersection requires to load both full lists before going for the intersection the access rules will fail. BUT if it tries to access all the posts from one list (the users/posts list) and then just the matching posts from /posts ... it might work. How does it actually work?

Is it a requirement of the firebase-util to have full read rights to the complete sets of intersecting resources?

Related to: #28, I can't seem to apply a query on the master index using the orderByValue() method. An 'undefined is not a function' error is returned, http://plnkr.co/edit/HaA1lz

Haven't stepped through the source yet although I've played around with some of the joins.

Couple questions about where the computations are happening.

Is this library just resolving a bunch of Firebase requests through a promise array or is a single payload returned for something like a join on multiple ID's?

I do joins of my own through service logic and I'm trying to figure out if I'm gaining any performance by using this utility.

Does this utility send a single payload and receive a single payload?... Or is this just a utility that makes it easier to compile promise arrays and join data once it all comes back?

Example:

Say I want to get all the messages from Jim's users object:

/*
 *    /users/-Ji28d73hd73h
 */

'name':'Jim',
'messages':{
  '-J8274629dj3':true,
  '-Ji27d7dh38d':true
}


/*
 *    /messages
 */

'-J8274629dj3':{
  text:'lol bro'
},
'-Ji27d7dh38d':{
  text:'u wot mate?'
}

Normally, I would loop through users.$id.messages and resolve the requests through a promise array. Then there would be some post processing to join the data on a new object.

• Does this library send a single, special request that let's Firebase know I'm looking for all of these records so multiple HTTP calls aren't made from loops?
• Is post processing for joins also done server side or does the library do this when the payload comes back?

I'm getting lots of deprecation warnings when using firebase-util now. Has any of its functionality been added into the new 2.0 release of the firebase web api so that I no longer need to use it?

Will there be a [firebase-release] with the fixes since May anytime soon? I would like to use them from the CDN/Bower. Thanks

Per #6:

npm install firebase/firebase-util -S

Is not working for me. The node_modules/firebase-util directory that gets created has /src, /test/ and /node_modules but no /dist. The firebase-util/package.json specifies its main as dist/firebase-util.min.js, which does not exist.

I have not tried the bower install suggested in #6, but I'm hoping to use firebase-util on both server and client.

on('child_changed',...) is being called many times when it probably shouldn't.

I'm opening four different issues because I'm not sure if they are related and, if they are not, they can be fixed separately. However, I used the same Plunker to simulate all of them, so it is a little bigger than usual. I tried to explain everything in the comments, but if something is not clear enough, just let me know.

http://plnkr.co/edit/Y0mp2XDtSQ0Pz9ClXctZ

Add this into JoinedRecord class. There is an internal bug in Firebase preventing once('value', ...) from correctly setting priorities. So this won't be possible until that is corrected.

When applying a limitToFirst() query on the main ref of a NormalizedCollection, the returned data is not restricted by the limit value, http://plnkr.co/edit/WLAkVp?p=preview

Also applies to limitToLast() query

I get a 404 error when I try to use the newest version from the CDN:
https://cdn.firebase.com/libs/firebase-util/0.2.3/firebase-util.js

Hi I am developing an app using ionic and I was using the infinite scroll from firebase-util. My problem is I want latest data first just like in twitter and then scroll down to older posts

I've tried applying query to the ref passed as argument to the NormalizedCollection and to the the NormalizedRef. In both cases I don't get any results.

Here are two plunks that illustrate this:

• Applying query to master index: http://plnkr.co/edit/max7TciVx48YG3cNcfpS
• Applying query to NormalizedRef: http://plnkr.co/edit/5AtkOtfvBorKkLnLCHAw

First - thank you! Firebase-util works nicely.

One issue that I've run into is utilizing limit() on my queries. The docs appear to tell me that I can use limit() and it won't throw an error if the limit() is applied to the reference before passed into firebase-util intersection (https://github.com/firebase/firebase-util/blob/master/src/join/README.md#queries-limit-startat-and-endat). Despite having done so, firebase-util throws the following error:

Error: Invalid path at position 0; it must be a valid Firebase or JoinedRecord instance, or if a props object is used, contain a ref key which is a valid instance.

Is use of limit() et al indeed possible?

Thanks!
irfaan

From your doc:

https://github.com/firebase/firebase-util/blob/master/src/join/README.md#queries-limit-startat-and-endat

var indexRef = new Firebase('URL/index_path').limit(10);
Firebase.util.intersection( indexRef, new Firebase('URL/data_path') );

And my code

                       var indexRef = new Firebase('path.com/'+subCategoryId).limit(10);
                                var join = Firebase.util.intersection(
                                          indexRef,
                                           fbRefs.groups().child(subCategoryId)
                                );
                      return $firebase(join);

Using

"firebase": "~1.0.13",
"angularfire": "~0.7.1",

Error:

Error: Invalid path at position 0; it must be a valid Firebase or JoinedRecord instance, or if a props object is used, contain a ref key which is a valid instance
at Object.PathLoader._assertValidPath (http://127.0.0.1:9000/bower_components/firebase-util/firebase-util.js:2870:22)
at Array.forEach (native)
Is there a bug or am I doing something wrong please?

Hi, i have a problem. I don't know why it does not work and when it does... it does not sync the data... maybe i am missing something, maybe there is a bug? Please look at these fiddle i have created for you.

This works:

http://jsfiddle.net/2D5JC/36/

But try to replace the external resources from angularfire 0.5 to

https://cdn.firebase.com/libs/angularfire/0.7.1/angularfire.min.js

and it will breaks. Only one status will be fetched + when i add new status to the DB the $scope.statuses is not updated.

Could you please explain it to me ? That would be awesome i have spent all day on it :(

But this isn't true. If exportVal() encounters a primitive with no priority, it will return the primitive. This probably breaks some of our parsing code in common/libs/utils.js

So that if one path fails, all paths are rolled back. This can't be atomic, but it can at least try to maintain data integrity except for the edge case where client goes offline between one set and another and never comes back (which would cause the same issues anyway). Could we use a counter/timestamp approach here?

Hi,
I got a warning/error when I try to mix multiple things (scroll, normalize, $firebaseArray)

var fb = new Firebase(FIREBASE_URL)
var ref = fb.child("users/user1/posts");
var postsRef = fb.child("posts");
var scrollRef = new Firebase.util.Scroll(ref,'$priority');
var norm = new Firebase.util.NormalizedCollection(scrollRef, postsRef).select('posts.from', 'posts.content', 'posts.time');
var userPosts = $firebaseArray(norm.ref());
userPosts.scroll = scrollRef.scroll;
return userPosts;

with the following structure :
{
users: {
user1: {
name: "Alice",
posts: {
post1: true,
post3: true,
... (let's assume there are 100 other posts)
}
},
user2: {
name: "Bob",
posts: {
post2: true
}
}
},
posts: {
post1: {
content: "Hi"
from: user1,
time: 1428933171010
},
post2: {
content: "Welcome",
from: user2,
time: 1428933171011
},
post3: {
content: "Ya"
from: user1,
time: 1428933171012
}
}
}

Here is the error/warning in JavaScript console : "this.off is not a function"

Ours are sadly incomplete and we could use a good number.

I've building an AngularFire project using browserify, and have installed Firebase-util via npm. I've required it like so:

var firebase = require('firebase');
var firebaseUtil = require('firebase-util');
var angularFire = require('angularfire');

And have included it as a dependency it in my angular.module() like this:

angular.module('outcomesApp', [
  'firebase',
  'firebaseUtil',
  ...
])

But for some reason I'm getting this error:

Uncaught Error: [$injector:modulerr] Failed to instantiate module outcomesApp due to:
Error: [$injector:modulerr] Failed to instantiate module firebaseUtil due to:
Error: [$injector:nomod] Module 'firebaseUtil' is not available! You either misspelled the module name or forgot to load it. If registering a module ensure that you specify the dependencies as the second argument.

Any idea why this isn't working for me? Any help is appreciated. Thanks in advance!

When applying the orderByValue() query on the main ref of a NormalizedCollection, the data does not sort by value. http://plnkr.co/edit/Zy1vax?p=preview

What is the right way to integrate this lib with Angular.js and AngularFire?

Is using $value as a field dependency supported? I'm running into issues when the value itself changes, with the following error:

FIREBASE WARNING: Exception was thrown by user callback. TypeError: Cannot read property 'ref' of null
    at Object.i.update (https://cdn.firebase.com/libs/firebase-util/0.2.4/firebase-util.min.js:9:22457)
    at https://cdn.firebase.com/libs/firebase-util/0.2.4/firebase-util.min.js:9:18531
    at https://cdn.firebase.com/js/client/2.2.3/firebase.js:49:642
    at Cb (https://cdn.firebase.com/js/client/2.2.3/firebase.js:46:165)
    at yb (https://cdn.firebase.com/js/client/2.2.3/firebase.js:22:216)
    at zb (https://cdn.firebase.com/js/client/2.2.3/firebase.js:21:1259)
    at Ih.h.Fb (https://cdn.firebase.com/js/client/2.2.3/firebase.js:202:287)
    at vh.h.Hd (https://cdn.firebase.com/js/client/2.2.3/firebase.js:189:266)
    at jh.Hd (https://cdn.firebase.com/js/client/2.2.3/firebase.js:180:364)
    at bh.tg (https://cdn.firebase.com/js/client/2.2.3/firebase.js:178:281) 
firebase.js:46 Uncaught TypeError: Cannot read property 'ref' of null

Here is the code I'm using:

var baseRef = new Firebase("https://dev.firebaseio.com");
new Firebase.util.NormalizedCollection(
  [baseRef.child("reservations"), "res"],
  [baseRef.child("reservations_guests"), "rg"],
  [baseRef.child("guests"), "g", "rg.$value"]
).select(
  {"key":"res.$value","alias":"reservation"},
  {"key":"rg.$value","alias":"guestId"},
  {"key":"g.$value","alias":"guest"}
).ref();

And here is a sample of what that database looks like:

reservations: {
  resId1: {
    status: 'arrived'
  },
  resId2: {
    status: 'outstanding'
  }
},
reservations_guests: {
  resId1: 'g140'
  resId2: 'g141'
},
guests: {
  g140: {
    name: 'Guest Number 1'
  },
  g141: {
    name: 'Guest Number 2'
  }
}

If I, for example, change reservations_guests/resId1 to g141, I get the error as shown above.

I'm following the Firebase-util Paginate Infinite Scroll demo, and was wondering if there is a way to load an initial number of items, instead of loading 0 items to begin with.

When I click on my button, I'm loading in the next 16 items. Is there a way I can load 16 items on the initial load, and then another 16 upon clicking my button, etc etc?

My HTML and JS looks like this:

index.html

<h3>Items loaded: {{scrollItems.length}}</h3>
<button ng-click="scrollItems.scroll.next(16)">Load Next 16</button>

app.js

app.controller('ServiceListController', ['$scope', '$firebaseArray', '$scrollArray',
  function($scope, $firebaseArray, $scrollArray) {
    var servicesRef = new Firebase('https://fbutil.firebaseio.com/paginate');
    $scope.scrollItems = $scrollArray(servicesRef, 'number');
}])

app.factory('$scrollArray', ['$firebaseArray', function($firebaseArray) {
  return function(ref, field) {
    var scrollRef = new Firebase.util.Scroll(ref, field);
    var list = $firebaseArray(scrollRef);
    list.scroll = scrollRef.scroll;

    return list;
  }
}])

Here's a demo of this in action.

Any help with this is appreciated. Thanks in advance!

Please consider making this a bower app.

Calling .filter() on NormalizedCollection doesn't filter out the results. Here's an example:
http://plnkr.co/edit/3WRXrvtkxsHMnxHX0DRe

Right now they only run in node.js, change the require args and such around so that they can be run from PhantomJS in node, and in any browser for public testing.

See #36 for an example of why these are needed. Adding removing listeners is difficult and can behave differently with complex combinations of add/remove on the same record.

I've tried to get it work in the past with no luck however. Here is how I
went about it, basically the example code from both the
normilizedCollection and the infiniteScroll put together.

function getAllPublicOddsOns() {
// grab the base url like normal
var fb = new Firebase(config.apiUrl);

// map the paths we are going to join
var norm = new Firebase.util.NormalizedCollection(
fb.child('oddsons'),
fb.child('public')
);

norm.select(
{ "key":"public.$value","alias":"public" },
"oddsons.description",
"oddsons.challengeUserID",
"oddsons.sentBy",
"oddsons.odds",
"oddsons.number",
"oddsons.oddsOnDetails",
"oddsons.challenger",
"oddsons.challengee"
);

// only show the oddsons that are public
norm = norm.filter(
function(data, key, priority) {
return data.public === true;
}
);

// get a reference we can use like a normal Firebase instance
var ref = norm.ref();

// create a scrollable reference
scrollRef = new Firebase.util.Scroll(ref, 'name');

// create a synchronized array on scope
$scope.allPublicOddsOnsList = $firebaseArray(scrollRef);
// load the first 10 public oddsons
scrollRef.scroll.next(1);
}

Where I'm getting this error:
http://puu.sh/gZBGZ/20fee211b9.png

When using filter(), on('child_added',...) and on('child_removed',...) are not working properly after child_changed on master index or on secondary path. Seems that filter() is not reapplied properly.

I'm opening four different issues because I'm not sure if they are related and, if they are not, they can be fixed separately. However, I used the same Plunker to simulate all of them, so it is a little bigger than usual. I tried to explain everything in the comments, but if something is not clear enough, just let me know.

http://plnkr.co/edit/Y0mp2XDtSQ0Pz9ClXctZ

Currently in $firebaseArray(ref).$add({...}).then(onFulfilled, onRejected);, the functions onFulfilled and onRejected are not called.

It's practical for me to happen so that I can get a reference to the item that was added by the user adding it. Currently I use $watch on the firebaseArray now, but I needed to build something special to filter out other events that where triggered by other users actions on the same array.

My code can become simpler if those oFulfilled and onRejected functions are called.

@katowulf Can you give me some pointers on where to implement this in firebase-util?

Here's a fiddle that compare push and $add: http://jsfiddle.net/devotis/869sgr01/2/

on('child_removed',...)'s snapshot does not contains the correct data.

• The snapshot contains only the data from the secondary path if the item is first removed from master index and is empty when it is removed from the secondary path. Maybe it should happen only once, with item's full data when it is removed from master index, and nothing else after that.
• The snapshot contains only the data from secondary path when remove() is used in NormalizedCollection's reference. It probably should have item's full data.
• The snapshot is empty if the item is first removed from secondary path. Maybe it was supposed to have a on('child_changed',...) showing the item without the data from the secondary path that was just removed and then a on('child_removed',...) showing the last state of the item when it was finally removed from the master index.

I'm opening four different issues because I'm not sure if they are related and, if they are not, they can be fixed separately. However, I used the same Plunker to simulate all of them, so it is a little bigger than usual. I tried to explain everything in the comments, but if something is not clear enough, just let me know.

http://plnkr.co/edit/Y0mp2XDtSQ0Pz9ClXctZ

When using filter(), on('value',...) is not working properly in most cases and, when not using filter(), sometimes it is called twice.

I'm opening four different issues because I'm not sure if they are related and, if they are not, they can be fixed separately. However, I used the same Plunker to simulate all of them, so it is a little bigger than usual. I tried to explain everything in the comments, but if something is not clear enough, just let me know.

http://plnkr.co/edit/Y0mp2XDtSQ0Pz9ClXctZ

Hello,

I decided to test the code provided on the firebase-util's pagination docs

var baseRef = new Firebase('https://fbutil.firebaseio.com/paginate');
var scrollRef = new Firebase.util.Scroll(baseRef, '$key');
// special scroll methods are namespaced in scrollRef.scroll
scrollRef.scroll.next(25);

And it throws this error when you run it:

Error: Firebase (2.2.3) INTERNAL ASSERT FAILED: Unrecognized index type!

Really weird error because if you pass in any other string as the orderBy field parameter, it runs smoothly.
I tried passing in '$key' and '$value' and it breaks every time.

I think that the Scroll function is issuing a orderByChild query every time you pass in a string as a parameter.
For example, if you pass in '$value', it throws this error:
Error: Query.orderByChild: "$value" is invalid. Use Query.orderByValue() instead.

Curiously enough, if you pass in '$priority' nothing happens.

Hi,

This new 0.2 version brings lot of new cool stuffs, thanks,

Is this possible (maybe in the future) to use NormalizedCollection with $firebaseArray ?

$firebaseObject seems to work but not the array.

There are no unit tests exercising limit/startAt/endAt functionality.

... resolved once all the child have been added / moved / removed

Don't know how to implement this feature nicely, but wanted to submit the idea :-)

thanks for this amazing lib !

firebase-util looks very useful, is anyone planning on porting it to iOS?

If the firebaseRef is firebase-Util reference -> There is an interesting "bug" with this code:
firebaseRef.on('value', function(snapshot) {
var data = snapshot.val();
data.id = snapshot.name(); // add the key as an id
});
The callback is trigger more times for one entry.