Retrospective: Building a Feed

As I shared a few weeks ago, I put together a small working prototype for a news feed which was based on some of our original thoughts around a developer-centric feed and I promised that I’d share some of my work and the progress.

The motive for sharing this kind of stuff is quite simple: Providing an inside look at not just what we’re building but how and the why of our project is important because it keeps us honest and it enables us to get great feedback. And it’s just what I’m used to doing when I put new things together. I learn as much via the process of sharing as I do creating!

What I ended up doing was putting together a technical retrospective for myself and the team as to what I thought about it (so far) so as to give some useful context around the work itself and what I believed could be leveraged moving forward. It’s worth reviewing this post first to get some foundation of why I started and how I went about putting it together.

Hopefully you find this useful in your early-stage product / concept and prototyping work as you attempt to maximize the very limited amount of time that you have to build and get stuff done.

(What you’ll see below is what I wrote internally as a technical retrospective. I’ve kept 99% of the post intact, exactly as I shared it, removing anything sensitive or completely useless for public consumption, which was very little, tbh.)

Tech Overview via Slack Docs / Notes
Tech Overview via Slack Docs / Notes

Intent

This is a quick technical overview of an experiment with the getstream.io system and API. My intent was to explore the use of the getstream platform as a possible tool for us to use as we build our own stream / feed.

Since it’s already built out with a few of the more typical features it would be a nice temporary solution as we move toward product/market fit. We’d be able to run tests and experiments with different features and feature-sets to determine interest from our first few batches of users.


TL;DR

I would continue to use it for early stages of our prototype but would definitely build one in-house as soon as we land on product/market fit. I’m wary on the scalability and would be very hesitant to build anything core on top of it.

Give it a spin: (link removed for safe-keeping… and for cost-savings!)

It’s a CRUD app. You can sign-up, login, post content, follow users, and customize a few things via a small admin dashboard.

GetStream.io isn’t terrible but it’s a much more “BETA” product than I had originally thought it was. It’s a young and “newer” service that still needs to be stress-tested from it’s own end. It has decent starting API but supports very limited social APIs and their call speed is a lot slower than I had hoped (and via their competition).


Tech Summary

Stream-js is the official JavaScript client for Stream, a web service for building scalable news feeds and activity streams.

This prototype app shows the news feeds and activity streams based on their APIs. This implementation (as requested by Jeff) is running Node.js. More specifically, it’s running Node.js for the backend and Express.js for the front-end.

The use of Node.js are well known and since it was requested I won’t explain any motives for its choosing, but, Express was useful because it provides a minimal interface for us to build out applications with very little overhead and a lot of flexibility. There are numerous modules available on NPM for Express, which can be plug-and-play.

For the database I’ve used MongoDB via mlab.com, deployed to Heroku.

For the design layer I used a design kit (Sketch file) provided via GetStream and a number of their out-of-the-box design elements. I choose the light-colored schema and used their “Option D” as found below:

Option D
Option D

Primary Language: JavaScript / Jade
API / WS Calls: GetStream.io API, location TBD
Framework: Express / Node.js
MVC: Yes
Dependencies: None
Target Platform: Web

As expressed above, this was a web-first implementation and any mobile implementations would come much later as we figure out features and gather feedback from the first initial users.

There is a number of views that have been created that you can see via the demo. You should be able to create new accounts (not verification on emails) and begin to post updates at will, follow existing people in the network, and get updated / curated feeds.

Here are some screenshots of the app:

Home View
Home View
Basic Account Management
Basic Account Management
List of Friends / Users
List of Friends / Users
Basic Sign Up
Basic Sign Up

API Usage via GetStream.io

This prototype was built to test and/or extend the existing API to get a feel for its viability as a prototype / v0 for our initial users / customers. GetStream built it “to scale” with a new app and that has yet to be determined. .

Below includes source code that interacts with their APIs:

1. Add Activities

//Add feed to timeline feedgroup for newsfeed. 
	var user1 = client.feed('timeline', userId); 
	user1.addActivity({
     actor: userId,
	 verb: 'add',
	 object: 'text:'+userId, 
	 message: feed_text, 
	 username: userName, 
	 userImage: userImage, 
	 email: email, 
	 feedImage: feedImage
}).then(function(data){ 
	console.log(data);
}).catch(function(reason){ 
	console.log(reason);
});

API:

     
// example 1 - starting point         

chris.addActivity({
	actor: 'chris',
	verb: 'add',
	object: 'picture:10',
	message: 'Some cool message here...'
});

2. Following Feeds

my_timeline = client.feed('timeline', myId); 
my_timeline.follow('user', followId);
console.log('timeline:' + myId + " follows user:" + followId);

API:

// Follow Minnie      
me = client.feed('timeline', 'me');      
me.follow('user', 'minnie');

3. Notification Feed

     
// Mark all activities as seen 
notificationFeed.get({limit:5, mark_seen:true}) 
.then(function(data) {
	console.log(data); 
})
.catch(function(reason) { 
	console.log(data); 
});

4. Reading Feed Followers

//---get my following--- 
	my_timeline.followers ({offset: 0, limit: 100})
	    .then((followers) => {
		var followers_list;
		followers_list = followers.results;
     
}).catch(function (err) { 
	console.error(err)
	req.flash('errors', err);
	res.redirect('/'); 
})

5. Reading Followed Feeds

//---get my following--- 
my_timeline.following({offset: 0, limit: 100})
 .then((following) => {
 var following_list;
 following_list = following.results;
}).catch(function (err) { 
console.error(err) 
req.flash('errors', err); 
res.redirect('/');     
})

6. Creating Feed Group

To create the feed groups the user needs to head to the Dashboard and create a necessary feed group there. User, Timeline, Notification, Timeline_aggregated

7. Retrieving Activities

var user1 = client.feed('timeline', userId); 
user1.get({limit:100, offset:0})
	.then(function(successData){ 
		console.log(successData); 
		res.render('home', {
			title: 'NewsFeed',
		    timeline: successData.results,
		     uploaded_file: req.session['uploaded_file'] 
		})
     }).catch(function(reason){
		console.log('retrieving data error:' + reason); 
		res.render('home', {
			title: 'NewsFeed',
				uploaded_file: req.session['uploaded_file']
		     }); 
	});

API:

me = client.feed('timeline', 'me'); 
activities = me.get({limit: 10});

8. UNFollowing Feeds

myId = req.body.myId; 
followId = req.body.followId;
     
my_timeline = client.feed('timeline', myId); 
my_timeline.unfollow('user', followId); 
console.log('timeline:' + myId + " follows user:" + followId);

Customized Parts via the API

There are a number of features that GetStream does not actively support at this point in time. For instance, “Global Feeds” is not something that they currently support (e.g. all feeds that were added to all groups).

9. Global Feeds

I pinged them about this and I received this response:

via matthisk (https://github.com/matthisk)

This is not supported we require you to store this information on your side. So for instance you would perform some sort of user management on your end. 

One user has certain feeds inside Stream feed groups which are referenced with their user id stored on your end. Now you can get all feeds stored in Stream by looping over all your users and using their user id's to create Stream feeds.

Consequently, I have built in a way to receive all feeds from every feed group and then I integrated them into one list to show Global Feeds List.

There is an issue here though: In case of users following specific users they will receive feeds from that follower. So, in a manual integration, a lot of the same feeds are being displayed now which is not optimal.

This is something that they should fix via their API or we should find another way to solve this obvious dilemma. I’m a bit surprised they haven’t fixed it or added to it already for some of these obvious needs.

10. Delete all Feeds from Feed Groups

I wanted a simple way to delete and “clear house” of the entire prototype but they do not support this via their API, which I’m not entirely surprised by. This can only be done via the GetStream dashboard and cannot be performed via API (as confirmed by their team).

KILL COMMAND! LOL.

Other Functions

Finally, here’s a list of other functions that were built that do not necessarily need a comprehensive break-down as many of these are self-evident:

  1. User Login (Email/Password)
  2. Forgot Password(Email)
  3. User Register (Email/Password)
  4. Uploading photos
  5. Changing Password
  6. Delete Account
  7. Delete All Account except oneself
  8. Edit Profile
  9. Post Feed (API)
  10. UI/UX from light weight front-end design via getstream.io
  11. NewsFeed (API)
  12. Global (API)
  13. My Timeline (API)
  14. Friends (API)
  15. Notifications (API)
  16. A small yet functional FAQ
  17. Retweet (API)

Finally, the getstream API dashboard is somewhat useful showing some visualizations of usage:

Usage Stats
Usage Stats
API Calls
API Calls

There were some more integrations possible for more analytics that I did not hook up:

Integration for analytics
Integration for analytics

More to come…


Also published on Medium.

Leave a Reply