Question about audio feature dimention?

[sailist]

Would you mind share your configuration of opensmile? I wonder how do you get audio feature with size 100.

[MatheusMMatos]

@sailist Did you got to replicate the other feature extractions?

[sailist]

There is no detailed explanation in this paper, in paper MMGCN (https://github.com/hujingwen6666/MMGCN), the author use opensmile with is10 configuration to extarct audio feature with size 1582. Visual and Text feature in MMGCN are also available.
btw, the code structure of the two is similar in some places....

[Coding511]

@sailist Did you extract the video features?

[sailist]

I use iemocap features provided by cogman and meld features provided by mmgcn.

[Coding511]

Okay, But are you able to train the network, as I am not, please guide me with my problem below:

When executing train.py , I am getting this error

No such file or directory: './model_checkpoints/model.pt'

[Coding511]

Do I need to run eval.py
the error is :

usage: eval.py [-h] --dataset {iemocap,iemocap_4,mosei}
[--data_dir_path DATA_DIR_PATH] [--device DEVICE]
[--modalities {a,at,atv}] [--emotion EMOTION]
eval.py: error: the following arguments are required: --dataset

[iabhinavjoshi]

Okay, But are you able to train the network, as I am not, please guide me with my problem below:

When executing train.py , I am getting this error

No such file or directory: './model_checkpoints/model.pt'

@Coding511
For training the network, try passing --from_begin argument to the training script. (The --from_begin argument trains the network from scratch, without expecting a pretrained model ('./model_checkpoints/model.pt') to train from)

python train.py --dataset="iemocap_4" --modalities="atv" --from_begin --epochs=55

Do I need to run eval.py the error is :

usage: eval.py [-h] --dataset {iemocap,iemocap_4,mosei} [--data_dir_path DATA_DIR_PATH] [--device DEVICE] [--modalities {a,at,atv}] [--emotion EMOTION] eval.py: error: the following arguments are required: --dataset

For evaluation, please refer to the google colab notebook referenced in the README.md file:

https://colab.research.google.com/drive/1biIvonBdJWo2TiYyTiQkxZ_V88JEXa_d?usp=sharing

[Coding511]

@iabhinavjoshi Thanks dear
But still, my device is not working may be due to lower computation.

Could you please guide me through 'CPU' installation or does Pytorch geometric don't work with GPU?
I have Cuda 10.2 but for torch-sparser it's saying a higher version is required, though I have successfully installed geometric.

or else how to train the model from scratch with your given features on colab if possible.
please give me step by step if possible on colab. I tried cloning this folder COGMEN and changing the directory to this.
and then without unzipping the model. After installing Install comet-ml.

%run train.py

But the error encounter is given below. Do I need to do anything else there?
Waiting for your reply @iabhinavjoshi .
Thanks

OSError                                   Traceback (most recent call last)
[/content/COGMEN/train.py](https://localhost:8080/#) in <module>()
      4 import torch
      5 import os
----> 6 import cogmen
      7 
      8 log = cogmen.utils.get_logger()

9 frames
[/content/COGMEN/cogmen/__init__.py](https://localhost:8080/#) in <module>()
      3 from .Dataset import Dataset
      4 from .Coach import Coach
----> 5 from .model.COGMEN import COGMEN
      6 from .Optim import Optim

[/content/COGMEN/cogmen/model/COGMEN.py](https://localhost:8080/#) in <module>()
      3 
      4 from .SeqContext import SeqContext
----> 5 from .GNN import GNN
      6 from .Classifier import Classifier
      7 from .functions import batch_graphify

[/content/COGMEN/cogmen/model/GNN.py](https://localhost:8080/#) in <module>()
      1 import torch.nn as nn
----> 2 from torch_geometric.nn import RGCNConv, TransformerConv
      3 
      4 
      5 class GNN(nn.Module):

[/usr/local/lib/python3.7/dist-packages/torch_geometric/__init__.py](https://localhost:8080/#) in <module>()
      2 from importlib import import_module
      3 
----> 4 import torch_geometric.data
      5 import torch_geometric.loader
      6 import torch_geometric.transforms

[/usr/local/lib/python3.7/dist-packages/torch_geometric/data/__init__.py](https://localhost:8080/#) in <module>()
----> 1 from .data import Data
      2 from .hetero_data import HeteroData
      3 from .temporal import TemporalData
      4 from .batch import Batch
      5 from .dataset import Dataset

[/usr/local/lib/python3.7/dist-packages/torch_geometric/data/data.py](https://localhost:8080/#) in <module>()
      1 from typing import (Optional, Dict, Any, Union, List, Iterable, Tuple,
      2                     NamedTuple, Callable)
----> 3 from torch_geometric.typing import OptTensor, NodeType, EdgeType
      4 from torch_geometric.deprecation import deprecated
      5 

[/usr/local/lib/python3.7/dist-packages/torch_geometric/typing.py](https://localhost:8080/#) in <module>()
      2 
      3 from torch import Tensor
----> 4 from torch_sparse import SparseTensor
      5 
      6 # Types for accessing data ####################################################

[/usr/local/lib/python3.7/dist-packages/torch_sparse/__init__.py](https://localhost:8080/#) in <module>()
     17     spec = cuda_spec or cpu_spec
     18     if spec is not None:
---> 19         torch.ops.load_library(spec.origin)
     20     else:  # pragma: no cover
     21         raise ImportError(f"Could not find module '{library}_cpu' in "

[/usr/local/lib/python3.7/dist-packages/torch/_ops.py](https://localhost:8080/#) in load_library(self, path)
    253             # static (global) initialization code in order to register custom
    254             # operators with the JIT.
--> 255             ctypes.CDLL(path)
    256         self.loaded_libraries.add(path)
    257 

[/usr/lib/python3.7/ctypes/__init__.py](https://localhost:8080/#) in __init__(self, name, mode, handle, use_errno, use_last_error)
    362 
    363         if handle is None:
--> 364             self._handle = _dlopen(self._name, mode)
    365         else:
    366             self._handle = handle

OSError: /usr/local/lib/python3.7/dist-packages/torch_sparse/_spmm_cuda.so: undefined symbol: _ZNK3c1010TensorImpl36is_contiguous_nondefault_policy_implENS_12MemoryFormatE
 

[sailist]

Would you mind share your configuration of opensmile? I wonder how do you get audio feature with size 100.

@iabhinavjoshi hi, would you mind explain this question?

[Coding511]

@sailist are you using their features directly?
What is the segmentation for training and testing? Are they using session 4 for testing?
What is this
choices=["iemocap", "iemocap_4", "mosei"]

For IEMOCAP could you guide me with the above problem, please?

[sailist]

@sailist are you using their features directly? What is the segmentation for training and testing? Are they using session 4 for testing? What is this choices=["iemocap", "iemocap_4", "mosei"]

For IEMOCAP could you guide me with the above problem, please?

This is not a dataset problem.

OSError: /usr/local/lib/python3.7/dist-packages/torch_sparse/_spmm_cuda.so: undefined symbol: _ZNK3c1010TensorImpl36is_contiguous_nondefault_policy_implENS_12MemoryFormatE

This exception means that you mismatched the version of torch and the dependences of torch_geometric, you can manully install them from https://pytorch-geometric.com/whl/

After installed right version, everything will be fine.

[Coding511]

@sailist, I am trying on my local desktop without GPU now.
and the code is working as the training started.
I will post the result soon.

However one of my questions was, How these features are segmented, I mean random split or session-wise, speaker-independent analysis done?

[Coding511]

Okay, But are you able to train the network, as I am not, please guide me with my problem below:
When executing train.py , I am getting this error
No such file or directory: './model_checkpoints/model.pt'

@Coding511 For training the network, try passing --from_begin argument to the training script. (The --from_begin argument trains the network from scratch, without expecting a pretrained model ('./model_checkpoints/model.pt') to train from)

python train.py --dataset="iemocap_4" --modalities="atv" --from_begin --epochs=55

Do I need to run eval.py the error is :
usage: eval.py [-h] --dataset {iemocap,iemocap_4,mosei} [--data_dir_path DATA_DIR_PATH] [--device DEVICE] [--modalities {a,at,atv}] [--emotion EMOTION] eval.py: error: the following arguments are required: --dataset

For evaluation, please refer to the google colab notebook referenced in the README.md file:

https://colab.research.google.com/drive/1biIvonBdJWo2TiYyTiQkxZ_V88JEXa_d?usp=sharing

Hi Abhinav, In which journal this paper is published?

[Coding511]

On my machine, your features are producing 82% classification accuracy. And I think in your paper, you are calming more than 84%.
What else do I need to consider here to reproduce your results?
Can I Email you?

[iabhinavjoshi]

Would you mind share your configuration of opensmile? I wonder how do you get audio feature with size 100.

@sailist Thank you for your interest. :)

We followed the scripts provided by the following repositories
https://github.com/soujanyaporia/multimodal-sentiment-analysis
https://github.com/declare-lab/multimodal-deep-learning

The audio feature size provided in the above repositories matches the one shared in our repo.
https://github.com/soujanyaporia/multimodal-sentiment-analysis/tree/master/dataset/iemocap/raw

They provide a detailed pipeline for feature extraction set along with the explanation for multiple datasets like MOSI, MOSEI, and IEMOCAP datasets.