Javascript

PostToTumblr v3.18 – Fixed Tumblr oauth change

head

 

Just a quick update to say I have now fixed the authentication issue my PostToTumblr that quite a few people contacted me about.

What was going on was that Tumblr appear to have changed the format of the data they return from a token request which was causing a library that PostToTumblr relies on to fail.

Before PostToTumblr can post content on a users behalf it first must get an “access token”. This token is given to PostToTumblr as part of the authentication flow.

When PostToTumblr first starts up it checks to see if it still has a valid token (they can expire over time and other various reasons). If it doesnt it must go through the authentication flow. Firstly it redirects the user to the grant permission dialog:

screenshot_02

When the user clicks allow Tumblr then returns an “oauth token” and an “oauth verifier” to PostToTumblr, which it can then use to get an “access token” which is used to do the posting.

The problem that this update fixed was that the “oauth verifier” that was returned from Tumblr changed:

screenshot_03

You see at the end of the query string there is now a “#_=_” well this was causing havoc with the URL parameter parsing code in the Google oauth library I was using.

My solution is quick and dirty, just strip out the “#_=_” from the url while parsing:

// MIKE HACK!!	  
if(param.indexOf('oauth_verifier=')!=-1) 
{
	param = param.replace('oauth_verifier=','');
	param = param.replace('#_=_','');		  
	decoded['oauth_verifier'] = ChromeExOAuth.fromRfc3986(param);
}
else
{	  	  
	var keyval = param.split("=");
	if (keyval.length == 2) {
		var key = ChromeExOAuth.fromRfc3986(keyval[0]);
		var val = ChromeExOAuth.fromRfc3986(keyval[1]);
		decoded[key] = val;
	}
}

Well I hope this helps anyone else that may encounter this issue too!

Recursive – Explore the endless web

Wow, well that took longer than expected! 44 days ago I blogged that I had started work on a second version of my Chrome Crawler extension and have only just managed to get it to a state I was happy with enough to release it. To be fair I had been on a trip to New York during that period so perhaps I can be excused. Having said that however I think the time has been well spent and am fairly proud of the result.

TL;DR

Recursive is an experimental tool for visualising the world wide web. Given a URL it downloads the page search for links and then recursively downloads those. The information is then displayed in a node-based graph.

The Name

So what’s this all about? Why is it called ‘Recursive’, why not ‘Chrome Crawler 2′?

Although I would like to have called the spiritual successor to ‘Chrome Crawler’, ‘Chrome Crawler 2′ Chrome’s branding guidelines forbid using the Chrome name or logo (they brought this in since the launch of Chrome Crawler 1).

With that in mind I decided that rather bend Chrome Crawler’s name and logo to fit the guidelines I would create a whole new logo, name and app. The app is a total rewrite from the previous iteration anyway so I thought it justified.

According to dictionary.com there is no definition for “Recursive” or “Recurse” but there is one for “Recursion”:

2. the application of a function to its own values to generate an infinite sequence of values.

So a tool that downloads pages, follows the links on that page to download other pages seemed like a rather apt description of something that is “Recursive”.

Video

Before I go much further, I put together this little video demonstrating some of the extensions core functionality:

Installing

Installing and upgrading is dead simple thanks to how Google Chrome’s extension system works. Just head over to this link and hit install:

https://chrome.google.com/webstore/detail/recursive/hbgbcmcmpiiciafmolmoapfgegbhbmcc

Then to launch it visit any website and hit the little icon in the Omnibox:

How it works

Recursive works by taking in a starting URL which it uses to download the page it points to:

Once that page is downloaded Recursive parses it looking for links and files. If it finds things it thinks are files then it records them against that URL. It then proceeds to visit all the links in turn, downloading the page then parsing the for yet more files and links.

This cycle continues until a certain “depth” is reached which is the maximum number of links away from the starting URL. You can set the maximum depth allowed in the settings:

One of the key improvements of Recursive over Chrome Crawler is the way it visualises the data as it is returned:

Every page is grouped by its domain and is represented by a circular “node”.

So for example “http://mikecann.co.uk/personal-project/tinkering-with-typescript/” would be grouped under the “mikecann.co.uk” domain. Any other pages found while running that match this domain are added as little page icons inside the host node.

Any files that are found on a given page are given an appropriate icon and added to that page’s domain node.

As Recursive downloads pages and follows links it records the path it takes. It then draws lines between the nodes that are linked:

Interacting

Using the mouse wheel you can zoom in and out to get a better perspective. Click and drag to move about the recursive space. You can also run the app in fullscreen if you so desire.

If you click on a node it tells Recursive to explore that node for one extra level depth.

Right clicking a node opens a menu that lets you either open all the pages contained in that node or view the files for that node.

Files

By using the context menu for a node you can checkout all the files that Recursive found for that node. The files are separated into various categories which you can toggle on or off:

Then if you wish you can download all the files as a zip.

The tech

I think i’ll leave this section for the next blog post as this one is long enough already.

If you would like specific info on the tech in the meantime however or have some suggestions for features then don’t hesitate to contact me: mike.cann@gmail.com

Well that’s about it hope you like it. I had a blast making it even if it did take alot longer than I was expecting. It doesn’t have much purpose really but there is a lot of cool new tech in there, which is reason enough to make it surely?

Hxaria, Infinite Terrain [HaXe, WebGL,dat.GUI]

So I have been working on my “Terraria like Terrain” project “Hxaria” again.

Following on from the last post, I have now made it so that each particle can have its texture changed. This completes the functionality required to render each tile as a point sprite, as talked about in my previous post.

The way it works is that the entire world is recorded in a 2×2 array Tilemap. This 2×2 array holds a single Tile object for every single tile in the world:

  1. class Tile
  2. {
  3. 	public var x : Int;
  4. 	public var y : Int;
  5. 	public var type : Int;
  6.  
  7. 	public function new(x:Int, y:Int, type:Int) { this.x = x; this.y = y; this.type = type; }
  8. }

 

When the TileRenderer needs to render it asks this Tilemap for a Tile that represents that screen coordinate, the Tilemap then offsets the position due to the camera movement and returns a tile. So it looks something like:

The tile type is then passed to the shader in attribute buffers per point sprite / tile along with all the tiles which are stored on a single texture:

The shader then performs the neccessary calculations to work out what the UV coordinate in the texture. The Vertex Shader:

  1. uniform float amplitude;
  2. uniform float tileSize;
  3. uniform float texTilesWide;
  4. uniform float texTilesHigh;
  5. uniform float invTexTilesWide;
  6. uniform float invTexTilesHigh;
  7.  
  8. attribute float size;
  9. attribute vec3 customColor;
  10. attribute float tileType;
  11.  
  12. varying vec3 vColor;
  13. varying vec2 vTilePos;
  14.  
  15. void main()
  16. {
  17. 	vColor = customColor;
  18.  
  19. 	float t = floor(tileType/texTilesWide);
  20. 	vTilePos = vec2(tileType-(t*texTilesWide), t); // +(.5/tileSize)
  21.  
  22. 	gl_PointSize = size;
  23. 	gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );
  24. }

 

And the Fragment Shader:

  1. uniform vec3 color;
  2. uniform sampler2D texture;
  3. uniform float invTexTilesWide;
  4. uniform float invTexTilesHigh;
  5.  
  6. varying vec3 vColor;
  7. varying vec2 vTilePos;
  8.  
  9. void main()
  10. {
  11. 	vec2 uv = vec2( gl_PointCoord.x*invTexTilesWide + invTexTilesWide*vTilePos.x, gl_PointCoord.y*invTexTilesHigh + invTexTilesHigh*vTilePos.y);	
  12.  
  13. 	gl_FragColor = texture2D( texture, uv );
  14. }

So it works in a way very much like a raster engine. You only have to render as many particles as the screen can contain.

If the screen area moves beyond the extent of the Tilemap then more tiles are randomly generated:

The new tiles are randomly selected from 4 different types, Dirt, Gold, Diamonds and Rock. I have added some controls to the demo that allow you to tweak these values to demonstrate the random tile generation:

The UI may look familiar to people that have seen any experiments anyone who has worked with Three.js before, its the very popular dat.GUI. Its a really simple library written in javascript for creating controls that can be used to tweak experiments, perfect for me!

To get dat.GUI to work with haxe, I used the awesome Extern feature of HaXe. This means that all I have to do is provide a stub interface to dat.GUI rather than a full implementation in haXe. This is great as it allows me to rapidly begin to use the library but also have the type safety of HaXe. It didnt take long to stub out the bits of the library I needed in an extern:

  1. package dat;
  2.  
  3. /**
  4.  * ...
  5.  * @author Mike Cann
  6.  */
  7.  
  8. extern class GUI
  9. {
  10.  
  11. 	public function new(options:Dynamic) : Void;
  12. 	public function add(options:Dynamic, name:String) : GUI;
  13. 	public function name(value:String) : GUI;
  14. 	public function min(value:Float) : GUI;
  15. 	public function max(value:Float) : GUI;
  16. 	public function step(value:Float) : GUI;
  17. 	public function onFinishChange(f:Void -> Void) : GUI;
  18. 	public function listen() : GUI;
  19. }

Then I used it like:

package ;
import dat.GUI;
 
/**
 * ...
 * @author
 */
 
class GUIManager
{
	public var goldChance : Float;
	public var rockChance : Float;
	public var diamondsChance : Float;
	public var mapWidth : Int;
	public var mapHeight : Int;
 
	private var gui : GUI;
	private var game : Game;
 
	public function new(game:Game)
	{
		this.game = game;
 
		gui = new GUI( { height : 5 * 32 - 1 } );
 
		goldChance = game.tilemap.goldSpawnChance;
		rockChance = game.tilemap.rockSpawnChance;
		diamondsChance = game.tilemap.diamondsSpawnChance;
		game.tilemap.mapResized = onTilemapResized;
		mapWidth = 0;
		mapHeight = 0;
 
		gui.add(this, 'goldChance').name("Gold").min(0).max(1).step(0.001).onFinishChange(function() { game.tilemap.goldSpawnChance = goldChance; } );
		gui.add(this, 'rockChance').name("Rock").min(0).max(1).step(0.001).onFinishChange(function() { game.tilemap.rockSpawnChance = rockChance; } );
		gui.add(this, 'diamondsChance').name("Diamond").min(0).max(1).step(0.001).onFinishChange(function() { game.tilemap.diamondsSpawnChance = diamondsChance; } );
		gui.add(this, 'mapWidth').listen();
		gui.add(this, 'mapHeight').listen();
	}
 
	private function onTilemapResized(mapW:Int, mapH:Int):Void
	{
		mapWidth = mapW;
		mapHeight = mapH;
	}
}

Simples!

Anyways you can check the final result out on this page: http://mikecann.co.uk/projects/hxaria/02/
(Click and drag to move the camera about)

I have also uploaded a quick video too:

I have also uploaded the source again to my github page: https://github.com/mikecann/Hxaria
(I have also created a tag, incase the source changes in the future)

Next up, lighting!

1 2 3  Scroll to top