Processing file: /Users/daurin/Projects/HyperPACE/field_data/DALEC/NF2405_VIIRS/raw/daily_files/DALEC_12_2024-05-31_174230Z.TXT
---- Initializing Data...
---- Reading Input File...
---- Processing Data...
... 0%
... 5%
... 10%
... 15%
... 20%
... 25%
... 30%
... 35%
... 40%
... 45%
... 50%
... 55%
... 60%
... 65%
... 70%
... 75%
... 80%
... 85%
... 90%
... 95%
Procesing: 100%
---- Writing Data to new files...
Traceback (most recent call last):
File "/Users/daurin/miniforge3/envs/hypercp/lib/python3.11/site-packages/pandas/core/algorithms.py", line 1870, in safe_sort
sorter = values.argsort()
^^^^^^^^^^^^^^^^
File "pandas/_libs/tslibs/timestamps.pyx", line 380, in pandas._libs.tslibs.timestamps._Timestamp.__richcmp__
TypeError: Cannot compare tz-naive and tz-aware timestamps
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/daurin/GitRepos/HyperCP/Source/raw_Data_Hourly_Cutter.py", line 130, in <module>
split_csv_hourly(args.file, args.output)
File "/Users/daurin/GitRepos/HyperCP/Source/raw_Data_Hourly_Cutter.py", line 101, in split_csv_hourly
for (date, hour), group in df.groupby(["date_group", "hour_group"]):
File "/Users/daurin/miniforge3/envs/hypercp/lib/python3.11/site-packages/pandas/core/groupby/ops.py", line 788, in get_iterator
splitter = self._get_splitter(data, axis=axis)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/daurin/miniforge3/envs/hypercp/lib/python3.11/site-packages/pandas/core/groupby/ops.py", line 799, in _get_splitter
ids, _, ngroups = self.group_info
^^^^^^^^^^^^^^^
File "pandas/_libs/properties.pyx", line 36, in pandas._libs.properties.CachedProperty.__get__
File "/Users/daurin/miniforge3/envs/hypercp/lib/python3.11/site-packages/pandas/core/groupby/ops.py", line 946, in group_info
comp_ids, obs_group_ids = self._get_compressed_codes()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/daurin/miniforge3/envs/hypercp/lib/python3.11/site-packages/pandas/core/groupby/ops.py", line 972, in _get_compressed_codes
group_index = get_group_index(self.codes, self.shape, sort=True, xnull=True)
^^^^^^^^^^
File "/Users/daurin/miniforge3/envs/hypercp/lib/python3.11/site-packages/pandas/core/groupby/ops.py", line 897, in codes
return [ping.codes for ping in self.groupings]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/daurin/miniforge3/envs/hypercp/lib/python3.11/site-packages/pandas/core/groupby/ops.py", line 897, in <listcomp>
return [ping.codes for ping in self.groupings]
^^^^^^^^^^
File "/Users/daurin/miniforge3/envs/hypercp/lib/python3.11/site-packages/pandas/core/groupby/grouper.py", line 621, in codes
return self._codes_and_uniques[0]
^^^^^^^^^^^^^^^^^^^^^^^
File "pandas/_libs/properties.pyx", line 36, in pandas._libs.properties.CachedProperty.__get__
File "/Users/daurin/miniforge3/envs/hypercp/lib/python3.11/site-packages/pandas/core/groupby/grouper.py", line 692, in _codes_and_uniques
codes, uniques = algorithms.factorize( # type: ignore[assignment]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/daurin/miniforge3/envs/hypercp/lib/python3.11/site-packages/pandas/core/algorithms.py", line 832, in factorize
uniques, codes = safe_sort(
^^^^^^^^^^
File "/Users/daurin/miniforge3/envs/hypercp/lib/python3.11/site-packages/pandas/core/algorithms.py", line 1883, in safe_sort
ordered = _sort_mixed(values)
^^^^^^^^^^^^^^^^^^^
File "/Users/daurin/miniforge3/envs/hypercp/lib/python3.11/site-packages/pandas/core/algorithms.py", line 1936, in _sort_mixed
nums = np.sort(values[~str_pos & ~none_pos])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/daurin/miniforge3/envs/hypercp/lib/python3.11/site-packages/numpy/core/fromnumeric.py", line 1017, in sort
a.sort(axis=axis, kind=kind, order=order)
File "pandas/_libs/tslibs/timestamps.pyx", line 380, in pandas._libs.tslibs.timestamps._Timestamp.__richcmp__
TypeError: Cannot compare tz-naive and tz-aware timestamps
Processing file: /Users/daurin/Projects/HyperPACE/field_data/DALEC/NF2405_VIIRS/raw/daily_files/DALEC_12_2024-06-01_125333Z.TXT
(hypercp) daurin@gs616-daurinm1:~/Projects/HyperPACE/field_data/DALEC/NF2405_VIIRS$
@mabdelmegidjr Most daily files for the cruise process okay, but DALEC_12_2024-05-31_174230Z.TXT (on the Box with configs) has a problem with timezone naive comparison at L101: