How Effective Is a Neural Network Model in Predicting Poker Hands?

AI Thread Summary
A neural network model outputs a vector of 169 probabilities representing potential poker hands based on player actions and situations. The challenge lies in testing the model's accuracy when each observation is derived from different distributions, complicating traditional validation methods like cross-validation. Suggestions include using a test statistic, such as Pearson's chi-square, to assess how well observed data fits the model's predicted distribution. The model's output is already a distribution, necessitating a method that evaluates single observations from each distribution. Accurate testing requires a careful approach to ensure the model's predictions align with actual outcomes.
tomeram
Messages
3
Reaction score
0
Hey

I have a neural network model that produces as an output a vector of 169 variables which represents the probabilty of having a certain hand in poker (2 random cards dealt from a regular deck - 169 possibilities if considering only if the cards are from same suit or not).
The model predict for each spesific situation and action made by a player in the game the distribution of having all possible hands.
I kept a random sample from the data for testing the model, and now I want to test it. Each row in the testing set contains the data of the situation, the action the player made and the hand he had, however the model produce a vector of 169 values (which represent the probabilty of having each of the possible hand).
I am looking for a statistical method to estimate the accuracy of the model - some kind of method that can say what is probabilty that the observation came from the distribution produced by the model.
Thanks
Tomer
 
Physics news on Phys.org
Cross-validation to test the accuracy of the model?

To test how well the observations fit the probability distribution produced by the model you could construct a test statistic (eg Pearson's chi square statistic) and repeatedly sample from your model's prob distribution to produce a distribution for the test statistic - this allows you to give a p-value (e.g. test statistic is greater than Z with y% probability) and you can then compare the value of the test statistic from your observations to see how well they fit the model.

Edit: I should say that in order for this to be reliable, you should be training the model on a different data set to the one you're later using to test its accuracy.
 
Hi
Thanks
The problem id that the output of the model is already a distribution - the distribution of getting a certain value. The problem is that each observation comes from a differnt observation and I need to know if the model predicts the distribution correctly. It is rare to get two point from the same observation, so I have to build a test based on single point from each distribution. I don't think cross validation will help this time.
 
Hi
Thanks
The problem id that the output of the model is already a distribution - the distribution of getting a certain value. The problem is that each observation comes from a differnt observation and I need to know if the model predicts the distribution correctly. It is rare to get two point from the same observation, so I have to build a test based on single point from each distribution. I don't think cross validation will help this time.
 

Thread 'Formal derivation of statement from Peano Arithmetic system'

I was reading a Bachelor thesis on Peano Arithmetic (PA). PA has the following axioms (not including the induction schema): $$\begin{align} & (A1) ~~~~ \forall x \neg (x + 1 = 0) \nonumber \\ & (A2) ~~~~ \forall xy (x + 1 =y + 1 \to x = y) \nonumber \\ & (A3) ~~~~ \forall x (x + 0 = x) \nonumber \\ & (A4) ~~~~ \forall xy (x + (y +1) = (x + y ) + 1) \nonumber \\ & (A5) ~~~~ \forall x (x \cdot 0 = 0) \nonumber \\ & (A6) ~~~~ \forall xy (x \cdot (y + 1) = (x \cdot y) + x) \nonumber...
View full post »

Thread 'Deductive proof in logic formal systems'

I was reading documentation about the soundness and completeness of logic formal systems. Consider the following $$\vdash_S \phi$$ where ##S## is the proof-system making part the formal system and ##\phi## is a wff (well formed formula) of the formal language. Note the blank on left of the turnstile symbol ##\vdash_S##, as far as I can tell it actually represents the empty set. So what does it mean ? I guess it actually means ##\phi## is a theorem of the formal system, i.e. there is a...
View full post »

Similar threads

I Linear regression and random variables
Replies
30
Views
4K
Neural networks as mathematical models of intelligence
Replies
4
Views
2K
B In dire Straights: Straights of 11+ w/ 2 Poker decks
Replies
7
Views
2K
Comp Sci Poker test: probability of Full House (confusion)
Replies
11
Views
2K
Poker and the shared pot at the table in a decentralised network
Replies
1
Views
2K
Issues with SLS Model for Rubber and Foam Compression Tests
Replies
5
Views
777
How to automate tests on a physical model without real data
Replies
4
Views
1K
Normalization condition with a neural network
Replies
3
Views
1K
Mathematica Can Python and R be used for data science techniques?
Replies
1
Views
2K
Python Help solving a geometrical matching issue with Graph Neural Networks
Replies
1
Views
2K

Hot Threads

Recent Insights

Back
Top