File_to_sub.txt

Q3. Learning rate decay problem
--->H-1,U-100,Lam-3,Learn-0.1  Check file of these only you will get the result
See Figures also
Learn	Hidden Layer	Hidden Unit 	Lambda	Tr_Loss		Tr_acc		Val_loss	Val_acc		Test_acc		
0.1	2		100		4	0.43304071	0.705173611	0.43465259	0.70125		0.50233
0.005	2		200		4	0.39843171	0.726458333	0.41648905	0.711875	0.48600
0.01	3		150		15	0.444501746	0.706840277	0.46146342	0.6925		0.53566
0.001	3		100		10	0.46473313	0.6778125	0.48018130	0.659375 	0.49200
0.01	3		200		2	0.41193493	0.70892361	0.42859600	0.6978125	0.47833
0.001	4		200		10	0.47738775	0.656319444	0.47311003	0.6671875	0.51400


Q4. 
Activation	Learn	H_Layer	H_Unit 	Lambda	Tr_Loss		Tr_acc		Val_loss	Val_acc		Test_acc
Tanh		0.001	3	100	10	0.47513541	0.70302083	0.4928550	0.6846875	0.50466
		0.001	2	300	10	0.62847272	0.69065979	0.6144809	0.6940625	0.48900

RELU		0.005	3	100	4	0.40617853	0.72135416	0.4200231	0.710625 	0.46033
		0.005	2	300	4	0.40377603	0.72246527	0.4065594	0.7215632	0.43633

SOFTPLUS	0.005	3	100	4	0.41564286	0.715 		0.4206334	0.716875	0.46600
		0.005	2	300	15	0.42695146	0.70291666	0.4073041	0.7215625	0.48766

Q5.
Drop out method
Activation	Learn	H_Layer	H_Unit 	Lambda	Tr_Loss		Tr_acc		Val_loss	Val_acc		Test_acc