On the error in examples

[okunoyukihiro2]

Dear PyXtal_FF developers

I'm a new to the PyXtal_FF, and I checked the examples in PyXtal_FF/examples.

I tried to run /examples/Si-Python/bp-NN.py

python3 bp-NN.py

the following error had occurred.

        ______       _    _          _         _______ _______ 
          (_____ \     \ \  / /        | |       (_______|_______)
           _____) )   _ \ \/ / |_  ____| |        _____   _____   
          |  ____/ | | | )  (|  _)/ _  | |       |  ___) |  ___)  
          | |    | |_| |/ /\ \ |_( ( | | |_______| |     | |      
          |_|     \__  /_/  \_\___)_||_|_(_______)_|     |_|      
                 (____/      


    A Python package for Machine Learning Interatomic Force Field
     Developed by Zhu's group at University of Nevada Las Vegas
The source code is available at https://github.com/qzhu2017/FF-project

=========================== version 0.1.8 =============================

Descriptor parameters:
type : BehlerParrinello
Rc : 5.0
cutoff : cosine

Features and precomputed descriptors exist: Si-BP/Train_db.dat

25 structures have been loaded.
Computing the descriptors...
Traceback (most recent call last):
File "bp-NN.py", line 40, in
ff.run(mode='train', TrainData=TrainData, TestData=TestData)
File "/usr/local/lib/python3.8/dist-packages/pyxtal_ff-0.1.8-py3.8.egg/pyxtal_ff/init.py", line 276, in run
testDB.store(TestData, self._descriptors, True, self.path+'ase.db')
File "/usr/local/lib/python3.8/dist-packages/pyxtal_ff-0.1.8-py3.8.egg/pyxtal_ff/utilities/init.py", line 104, in store
self.add(function, data)
File "/usr/local/lib/python3.8/dist-packages/pyxtal_ff-0.1.8-py3.8.egg/pyxtal_ff/utilities/init.py", line 140, in add
for i, d in enumerate(p.imap_unordered(func, _data)):
File "/usr/lib/python3.8/multiprocessing/pool.py", line 868, in next
raise value
File "/usr/lib/python3.8/multiprocessing/pool.py", line 537, in _handle_tasks
put(task)
File "/usr/lib/python3.8/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/usr/lib/python3.8/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
TypeError: cannot pickle '_dbm.dbm' object

Same errors also occurred in SiO2-Python.

I had checked unit test of pyxtal_FF (python pyxtal_ff/test_all.py ) and in this case, errors did't occurred.

My environment are
Python 3.8.10
numba 0.54.1
numpy 1.20.3
torch 1.10.0
monty 2021.8.17
seekpath 2.0.1
ase 3.22.0

What is wrong for my environment ?

Sincerely,

Yukihiro Okuno.

[qzhu2017]

@okunoyukihiro2

Sorry, I just realized that I never got notification about this issue. If you are still interested in using PyXtal_FF, let me know.

[okunoyukihiro2]

Thank you for your reply.
I had changed the environment and there are no more error, (But I cannot resolve the reason
for the first error. ）
However, when I started the SiO2-Python/python so4-full.py, an error occurs

--------- error message ------

Descriptor parameters:
type : Bispectrum
Rc : 4.9
cutoff : cosine
lmax : 3
normalize_U : False

Traceback (most recent call last):
File "so4-full.py", line 32, in
ff.run(mode='train', TrainData=train_data)
File "/home3/okuno/.conda/envs/torch/lib/python3.8/site-packages/pyxtal_ff-0.1.8-py3.8.egg/pyxtal_ff/init.py", line 265, in run
trainDB.store(TrainData, self._descriptors, True, self.path+'ase.db')
File "/home3/okuno/.conda/envs/torch/lib/python3.8/site-packages/pyxtal_ff-0.1.8-py3.8.egg/pyxtal_ff/utilities/init.py", line 93, in store
data = parse_OUTCAR_comp(structure_file)
File "/home3/okuno/.conda/envs/torch/lib/python3.8/site-packages/pyxtal_ff-0.1.8-py3.8.egg/pyxtal_ff/utilities/init.py", line 554, in parse_OUTCAR_comp
raise ValueError("Cannot identify the line starts with energy without entropy")
ValueError: Cannot identify the line starts with energy without entropy

---

It seems that error has parser for OUTCAR_comp.

By the way, I'm now try to make machine learning potential for amorphous composed of '4' species.
And , I had prepared xsf files, and want to convert json structure for Pyxtal_FF.
Are there any converter crystal structures file format to json for PyXtal_FF ?

Sincerely.

[qzhu2017]

@okunoyukihiro2
For SiO2, we rely on the OUTCAR from external url link. It looks like this file has been modified. Therefore, the script no longer works.

If you have xsf files, I suggest you convert the structure to ase.db format. For pyxtal_FF, the most convenient format would be ase.db. If you want, we can schedule a quick video meeting. I can show you how to make the conversion.

[okunoyukihiro2]

Thank you for your reply. 　I had just make script for converting the xsf format to json format, where the
json structure are given below

json={'formula':formula,
'coords':
{'@module':'numpy',
'@Class' :'array',
'dtype' :'float64',
'data': coords},
'lattice':
{'@module':'numpy',
'@Class':'array',
'dtype':'float64',
'data':cell},
'elements':elems,
'energy':energy,
'time' : time,
'force': forces,
}

I will try to make ase.db from xsf file in according to the source code of utility/init.py

Here, we also found another problem. I tried to make potential for '3-' species amorphous
structure with SOAP or SO3 descriptors, it required tremendous memory during the
pytorch training, and the calculation were not proceeded.
I prepared 10000 structure where each crystal structure contain about 100 atoms, and
the input parameters we set for the training

train_data = "data_xsf_training.json"
descriptors = {'type': 'SOAP',
'Rc': 5.0,
'parameters': {'lmax': 4, 'nmax': 3},
'force':True,
'stress':False,
'N_train': 10000,
#'N_test':
}
model = {'system': ['A','B','C'],
'hiddenlayers': [20, 20],
#'restart': 'A-B-C-SOAP/20-20-checkpoint.pth',
'epoch' : 1000,
'memory': 'out',
'batch_size':500
}

How should we treat the memory problems ? I want to include the 'force' in
the training for sufficiently learning of potential energy surface.

Sincerely.

[qzhu2017]

@okunoyukihiro2

Well. There is a way to work around it. If you can split it your 10K structure into 10 sets. And then train each of them sequentially. I will post some sample scripts tomorrow when I have time.

[okunoyukihiro2]

Thank you for your reply. If I can get sample, I will be very glad.
(Or should I change the descriptor from SO3 to weighted ACSF where the size of dimension for descriptor
becomes smaller? )

Sincerely.

[okunoyukihiro2]

Dear qzhu2017.

If you have time, I expect the upload of the scripts which avoid large memory use for training.

Sincerely.

[qzhu2017]

@okunoyukihiro2

Sorry for the delay.

I just added a new example to illustrate the idea of multiple batch. So you just need split your original json files into many subfiles. PyXtal_FF will train the model by iteratively scanning each subdata set.

https://github.com/qzhu2017/PyXtal_FF/tree/master/examples/Si-multi

I just tried on my own dataset, it seems resolve the memory issue on my end. Could you please give it a try and let me know if it works.

[okunoyukihiro2]

Thank you very much for script and teaching me on the
training method.

I will try it and show the result on this board.

Sincerely,

Yukihiro Okuno