Alway profit 0 when evaluating a model

[Kalelv45]

It always stays at profit 0.00 after I evaluate some model.
Why can this be?

[kkuette]

Your agent did not learn well, i think it's just overfitting.

[iorobot]

Look at #1.
I also have that problem but even with that solution my results using almost the same model(model_1000 or model_1010) are extremely volatile.

[kkuette]

Try to tweak your settings.
if you want something with more functionalities you can look at this

[satinder147]

@Kalelv45 @kkuette @iorobot @edwardhdlu.I tried it myself and found that tweaking the code a little bit yields very good results. I changed the way reward was provided. If there were more that 20 consecutive buys or 50 consecutive "no action", then I gave the agent a big negative reward like -500. Apart from this I removed the max(in the buy action) and multiplied the reward by 100. I trained the model for 220 episodes and the 220th showed a profit of 25000 dollars on the last three year google stock price data.

[iorobot]

Nice!Can you pull the code?

[chmbrs]

@Kalelv45 @kkuette @iorobot @edwardhdlu.I tried it myself and found that tweaking the code a little bit yields very good results. I changed the way reward was provided. If there were more that 20 consecutive buys or 50 consecutive "no action", then I gave the agent a big negative reward like -500. Apart from this I removed the max(in the buy action) and multiplied the reward by 100. I trained the model for 220 episodes and the 220th showed a profit of 25000 dollars on the last three year google stock price data.

You mean that you removed the max function in the sell action, right?

[kkuette]

Are you sure that your model isnt overfitted ?

[satinder147]

I removed the max function because otherwise the model is not penalized for wrong decisions. Then you can increase the size of the penalty(not everytime) by multiplying the reward it gets after selling by 100.
As far as overfitting is concerned, I have tried it with a bunch of other stocks and it is working really good.

[gucciwang]

What's the window size you're using? @satinder147

[satinder147]

@calvin-is-seksy 10

[satinder147]

I just made an pull request. You all can have a look

[iorobot]

Even with your code i still have 0 profit with the GSPC dataset. Any ideas why?

[gucciwang]

I used a window size of 20, and found that my model converges and profits the most on my 30th episode. All models after 100 seems to diverge and just never buy anything. Any ideas on how to tweak the model to further converge?

[iorobot]

well, it seems to me that is only a coincidence, have you try to use different data?Do you find the same behaviour?

[GarfieldHuang]

I used a window size of 20, and found that my model converges and profits the most on my 30th episode. All models after 100 seems to diverge and just never buy anything. Any ideas on how to tweak the model to further converge?

hello, I want to know how to plot this graph??