Preheat oven to 180 C. Grease two 23cm round cake tins.
In a large bowl, beat brown sugar, oil, egg and vanilla together until smooth. Combine flour, salt and bicarbonate of soda. Add to sugar mixture alternately with milk. Beat until smooth. Stir in rhubarb and flaked almonds. Pour into prepared pans.
To make the Topping; in a small bowl, Combine caster sugar and butter in a small bowl. Stir in flaked almonds. Sprinkle topping over mixture in cake tins.
Bake in preheated oven for 30 to 35 minutes or until the cake springs back when you touch it.