Results replication mentioned in the paper

[aks1207]

Hi,

I read the paper and it was a great read. I have a couple of questions, namely

1. I tried running the experiment on GasSensorData using the below mentioned parameters

• batch_size=200
• latent_size=128
• learning_rate=0.001
• net_type='mlp'
• num_epochs=100
• num_folds=5
• num_layers=3
• oodclass_idx=0
• reg_lambda=1.0

and got the results as

== Final results ==
The best ID accuracy on "data_gas" test samples :  96.14 ( 1.163)
The best OOD accuracy on "class_0" test samples :
 TNR85           TNR95           AUROC           DTACC           AUIN            AUOUT          
 49.22 ( 4.726)  30.70 ( 6.331)  71.50 ( 1.772)  67.74 ( 2.368)  54.77 ( 1.536)  81.01 ( 1.656). 

But the paper mentions better results. Can you kindly help me if I have missed something in the experimentation ?

2. In the paper one of the baselines is Mahalnobis Distance utilisation. Can you eloborate on how it is being used ?

• Is it being used for distance calculation instead of equation 4 mentioned in the paper ?
• If yes, Is the mahalnobis distance covariance matrix is calculated using all data points representation that belong to a particular class?
• What is the covariance matrix during testing phase as we donot know what class each point belongs to? Do we still use the one built from using training samples ?
• Does the score function utilise this Mahalnobis distance and is it then followed by push and pull loss ?
• If not, what is the setup for Mahalnobis distance based baseline ?

3.In section 3.1.1 it was mentioned that BCD (Block coordinate distance is used for optimisation), but I could see mini batch gradient descent with Adam optimiser in the code. Is my interpretation wrong? If so, kindly correct me.