The fairest thing is either to have a flat tax rate OR tax each income stream separately. After all it is not fair to tax your combined income when you have all positive income streams but then not allow you to reduce that combined income when you happen to have losses on one or more of those income streams.
