Parallel HDF5: Try fixing strict collective requirements of HDF5 >= 2.0#1862
Parallel HDF5: Try fixing strict collective requirements of HDF5 >= 2.0#1862franzpoeschel wants to merge 91 commits intoopenPMD:devfrom
Conversation
franzpoeschel
force-pushed
the
fix-parallel-hdf5
branch
2 times, most recently
from
11ba7aa to
b6ea4b4
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| while (!(*m_handler).m_work.empty()) | ||
| { | ||
| IOTask &i = (*m_handler).m_work.front(); | ||
| // verifyFlushType(i.operation, l); |
Check notice
Code scanning / CodeQL
Commented-out code Note
franzpoeschel
force-pushed
the
fix-parallel-hdf5
branch
2 times, most recently
from
a5c1985 to
bcb55ef
Compare
| { | ||
| template <typename T, typename RecordType> | ||
| PostProcessConvertedAttributeImpl<T, RecordType>:: | ||
| PostProcessConvertedAttributeImpl(RecordType record_in, handler_t reader_in) |
Check notice
Code scanning / CodeQL
Large object passed by value Note
| REQUIRE(r.numAttributes() == 0); | ||
| // TODO: unitSI | ||
| // REQUIRE(r["x"].numAttributes() == 0); | ||
| // REQUIRE(r["y"].numAttributes() == 0); |
Check notice
Code scanning / CodeQL
Commented-out code Note test
This makes it easier to keep MPI processes in sync
These were unnecessary, but they snuck WRITE_ATT tasks into the skeleton flush.
not so great, but lets keep that for now
This reverts commit 6a5c9f5.
franzpoeschel
force-pushed
the
fix-parallel-hdf5
branch
from
48b4c2a to
07fb558
Compare
| // if (access::write(IOHandler()->m_frontendAccess)) | ||
| // { | ||
| // populateDefaultMetadata(); | ||
| // } |
Check notice
Code scanning / CodeQL
Commented-out code Note
| } | ||
| } // namespace | ||
|
|
||
| std::future<void> AbstractIOHandlerImpl::flush(FlushLevel l) |
Check warning
Code scanning / CodeQL
Poorly documented large function Warning
There was a problem hiding this comment.
CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
1 participant
It seems that HDF5 has become quite a bit pickier about metadata definitions in parallel setups with versions 2.0 and 2.1, leading to hangups.
Earlier, it was enough to define them consistently across ranks, now we apparently have to keep the exact same order of operations.
This is bad for the Span API which runs internal flushes for structure setup.
resetDataset()flushParticlesPathandflushMeshesPathfunctions, these unnecessarily leaked attribute flushes into the structure setup