I addressed this above. Material progress does lead to happiness, if by no other reason, then because it removes some of the causes of unhappiness.
It is hardly reasonable to blame this on physics and chemistry. The mistreatement of factory workers was hardly anything new when we consider how the old aristocracy treated the peasants during the earlier eras, and all the advances in weaponry only gave the old aristocracy more sophisticated tools to enact their agenda. Warfare and atrocities have unfortunately been the norm for as long as humanity has existed. The fact that the west developed the best tools for this can be attributed to the natural sciences, but the will to use them for evil ends can clearly not be ascribed to them.
In fact, this is actually an example of the dark side of human nature and society. While it may sound callous, the hard truth is you have to take the good with the bad. Nearly every advance can be used for good and for evil, but that doesn't change the fact that the natural sciences have contributed far more good than bad advances.