Note
Click here to download the full example code
Learning the fundamentals of pynapple
Learning objectives
- Instantiate the pynapple objects
- Make the pynapple objects interact
- Use numpy with pynapple
- Slicing pynapple objects
- Learn the core functions of pynapple
- Extras : pynajax
The pynapple documentation can be found here.
The documentation for objects and method of the core of pynapple is here.
Let's start by importing the pynapple package and matplotlib to see if everything is correctly installed.
If an import fails, you can do !pip install pynapple matplotlib in a cell to fix it.
import pynapple as nap
import matplotlib.pyplot as plt
import numpy as np
For this notebook we will work with fake data. The following cells generate a set of variables that we will use to create the different pynapple objects.
var1 = np.random.randn(100) # Variable 1
tsp1 = np.arange(100) # The timesteps of variable 1
var2 = np.random.randn(100, 3) # Variable 2
tsp2 = np.arange(0, 100, 1) # The timesteps of variable 20
col2 = ['potato', 'banana', 'tomato'] # The name of each columns of var2
var3 = np.random.randn(1000, 4, 5) # Variable 3
tsp3 = np.arange(0, 100, 0.1) # The timesteps of variable 3
random_times_1 = np.array([3.14, 37.0, 42.0])
random_times_2 = np.array([10, 25, 50, 70])
random_times_3 = np.sort(np.random.uniform(10, 80, 100))
starts_1 = np.array([10000, 60000, 90000]) # starts of an epoch in `ms`
ends_1 = np.array([20000, 80000, 95000]) # ends in `ms`
Instantiate pynapple objects
This is a lot of variables to carry around. pynapple can help reduce the size of the workspace. Here we will instantiate all the different pynapple objects with the variables created above.
Let's start with the simple ones.
Question: Can you instantiate the right pynapple objects for var1, var2 and var3? Objects should be named respectively tsd1, tsd2 and tsd3. Don't forget the column name for var2.
tsd1 = nap.Tsd(t=tsp1, d=var1)
tsd2 = nap.TsdFrame(t=tsp2, d=var2, columns = col2)
tsd3 = nap.TsdTensor(t=tsp3, d=var3)
Question: Can you print tsd1?
print(tsd1)
Out:
Time (s)
---------- ----------
0.0 -1.84256
1.0 1.14731
2.0 -0.96527
3.0 1.2207
4.0 -1.23454
5.0 -0.0869564
6.0 1.28998
...
93.0 0.120654
94.0 -1.71558
95.0 -0.932256
96.0 1.09668
97.0 -0.677977
98.0 1.60345
99.0 -2.08056
dtype: float64, shape: (100,)
Question: Can you print tsd2?
print(tsd2)
Out:
Time (s) potato banana tomato
---------- -------- -------- --------
0.0 -1.41454 0.65121 -0.71037
1.0 -0.25722 -0.83404 -0.0936
2.0 0.83082 1.62686 1.08766
3.0 -0.22232 -0.07435 0.71038
4.0 -0.57767 -0.85094 -0.48202
5.0 -0.024 0.0059 3.41439
6.0 -0.2075 0.25666 0.33251
...
93.0 1.35996 -1.06328 1.05836
94.0 -0.80419 1.49848 0.41703
95.0 -1.03541 -0.26662 -0.16504
96.0 1.86061 0.43365 -1.09489
97.0 1.4186 -1.73365 0.50641
98.0 -0.4638 0.96742 -0.56946
99.0 -0.30446 -0.55539 0.10715
dtype: float64, shape: (100, 3)
Question: Can you print tsd3?
print(tsd3)
Out:
Time (s)
---------- -------------------------------
0.0 [[-2.759185 ... 0.338431] ...]
0.1 [[ 0.828552 ... -0.973802] ...]
0.2 [[-1.395534 ... -0.234255] ...]
0.3 [[-2.091205 ... -0.489731] ...]
0.4 [[-1.333882 ... 0.383066] ...]
0.5 [[ 1.242666 ... -0.305255] ...]
0.6 [[ 0.336797 ... -1.949168] ...]
...
99.3 [[-0.867444 ... 1.17642 ] ...]
99.4 [[0.307781 ... 1.985537] ...]
99.5 [[ 0.13366 ... -0.670037] ...]
99.6 [[0.695887 ... 1.224848] ...]
99.7 [[-1.243469 ... -0.962183] ...]
99.8 [[ 1.610111 ... -1.162245] ...]
99.9 [[-1.273797 ... 0.432683] ...]
dtype: float64, shape: (1000, 4, 5)
Question: Can you create an IntervalSet called ep out of starts_1 and ends_1 and print it? Be careful, times given above are in ms.
ep = nap.IntervalSet(start=starts_1, end=ends_1, time_units='ms')
print(ep)
Out:
start end
0 10 20
1 60 80
2 90 95
shape: (3, 2), time unit: sec.
The experiment generated a set of timestamps from 3 different channels.
Question: Can you instantiate the corresponding pynapple object (ts1, ts2, ts3) for each one of them?
ts1 = nap.Ts(t=random_times_1)
ts2 = nap.Ts(t=random_times_2)
ts3 = nap.Ts(t=random_times_3)
This is a lot of timestamps to carry around as well.
Question: Can you instantiate the right pynapple object (call it tsgroup) to group them together?
tsgroup = nap.TsGroup({0:ts1, 1:ts2, 2:ts3})
Question: ... and print it?
print(tsgroup)
Out:
Index rate
------- -------
0 0.03954
1 0.05272
2 1.31801
Interaction between pynapple objects
We reduced 12 variables in our workspace to 5 using pynapple. Now we can see how the objects interact.
Question: Can you print the time_support of tsgroup?
print(tsgroup.time_support)
Out:
start end
0 3.14 79.0122
shape: (1, 2), time unit: sec.
The experiment ran from 0 to 100 seconds and as you can see, the TsGroup object shows the rate. But the rate is not accurate as it was computed over the default time_support.
Question: can you recreate the tsgroup object passing the right time_support during initialisation?
tsgroup = nap.TsGroup({0:ts1, 1:tsd2, 2:ts3}, time_support = nap.IntervalSet(0, 100))
Question: Can you print the time_support and rate to see how they changed?
print(tsgroup.time_support)
print(tsgroup.rate)
Out:
start end
0 0 100
shape: (1, 2), time unit: sec.
0 0.03
1 1.00
2 1.00
Name: rate, dtype: float64
Now you realized the variable tsd1 has some noise. The good signal is between 10 and 30 seconds and 50 and 100.
Question: Can you create an IntervalSet object called ep_signal and use it to restrict the variable tsd1?
ep_signal = nap.IntervalSet(start=[10, 50], end=[30, 100])
tsd1 = tsd1.restrict(ep_signal)
You can print tsd1 to check that the timestamps are in fact within ep.
You can also check the time_support of tsd1 to see that it has been updated.
print(tsd1)
print(tsd1.time_support)
Out:
Time (s)
---------- ---------
10.0 -1.31888
11.0 -0.718113
12.0 1.24419
13.0 -0.830932
14.0 -0.594185
15.0 -1.8992
16.0 -0.35161
...
93.0 0.120654
94.0 -1.71558
95.0 -0.932256
96.0 1.09668
97.0 -0.677977
98.0 1.60345
99.0 -2.08056
dtype: float64, shape: (71,)
start end
0 10 30
1 50 100
shape: (2, 2), time unit: sec.
Numpy & pynapple
Pynapple objects behaves very similarly like numpy array. They can be sliced with the following syntax :
tsd[0:10] # First 10 elements
Arithmetical operations are available as well :
tsd = tsd + 1
Finally numpy functions works directly. Let's imagine tsd3 is a movie with frame size (4,5).
Question: Can you compute the average frame along the time axis using np.mean and print the result?
print(np.mean(tsd3, 0))
Out:
[[ 0.00369424 -0.04036717 0.05846582 0.01814495 0.03126356]
[-0.01111499 0.01897265 0.01847203 -0.03588318 -0.01946324]
[-0.00429385 -0.00771617 -0.04040585 -0.01595809 0.01454269]
[ 0.03191239 0.02436548 -0.01644701 -0.02614312 0.03799712]]
Question:: can you compute the average of tsd2 for each timestamps and print it?
print(np.mean(tsd2, 1))
Out:
Time (s)
---------- ----------
0.0 -0.491235
1.0 -0.394953
2.0 1.18178
3.0 0.137901
4.0 -0.636877
5.0 1.13209
6.0 0.127224
...
93.0 0.451681
94.0 0.370439
95.0 -0.489025
96.0 0.399792
97.0 0.063786
98.0 -0.0219442
99.0 -0.250899
dtype: float64, shape: (100,)
Notice how the output in the second case is still a pynapple object. In most cases, applying a numpy function will return a pynapple object if the time index is preserved.
Slicing pynapple objects
Multiple methods exists to slice pynapple object. This parts reviews them.
IntervalSet also behaves like numpy array.
Question: Can you extract the first and last epoch of ep in a new IntervalSet?
print(ep[[0,2]])
Out:
start end
0 10 20
1 90 95
shape: (2, 2), time unit: sec.
Sometimes you want to get a data point as close as possible in time to another timestamps.
Question: Using the get method, can you get the data point from tsd3 as close as possible to the time 50.1 seconds?
print(tsd3.get(50.1))
Out:
[[-0.51814384 -1.24641445 -0.86150421 -0.15583572 0.24216516]
[ 0.24512378 -0.06590823 -1.56252411 -1.04071667 -0.57383551]
[ 0.22999762 -0.90951171 -0.83827371 -1.33318721 -1.47092034]
[ 2.46506458 -0.35632779 -1.05628497 0.7909005 0.06001689]]
TsGroup manipulation
TsGroup is under the hood a python dictionnary but the capabilities have been extented.
Question: Can you run the following command tsgroup['planet'] = ['mars', 'venus', 'saturn']
tsgroup['planet'] = ['mars', 'venus', 'saturn']
Question: ... and print it?
print(tsgroup)
Out:
Index rate planet
------- ------ --------
0 0.03 mars
1 1 venus
2 1 saturn
After initialization, metainformation can only be added. Running the following command will raise an error: tsgroup[3] = np.random.randn(3).
From there, you can slice using the Index column (i.e. tsgroup[0]->nap.Ts, tsgroup[[0,2]] -> nap.TsGroup).
But more interestingly you can also slice using the metadata. There are multiple methods for it : getby_category, getby_threshold, getby_intervals.
Question: Can you select only the elements of tsgroup with rate below 1Hz?
tsgroup.getby_threshold("rate", 1, "<")
tsgroup[tsgroup.rate < 1.0]
Out:
Index rate planet
------- ------ --------
0 0.03 mars
Core functions of pynapple
This part focuses on the most important core functions of pynapple.
Question: Using the count function, can you count the number of events within 1 second bins for tsgroup over the ep_signal intervals?
count = tsgroup.count(1, ep_signal)
Pynapple works directly with matplotlib. Passing a time series object to plt.plot will display the figure with the correct time axis.
Question: In two subplots, can you show the count and events over time?
plt.figure()
ax = plt.subplot(211)
plt.plot(count, 'o-')
plt.subplot(212, sharex=ax)
plt.plot(tsgroup.restrict(ep_signal).to_tsd(), 'o')
Out:
[<matplotlib.lines.Line2D object at 0x7f8744d465d0>]
From a set of timestamps, you want to assign them a set of values with the closest point in time of another time series.
Question: Using the function value_from, can you assign values to ts2 from the tsd1 time series and call the output new_tsd?
new_tsd = ts2.value_from(tsd1)
Question: Can you plot together tsd1, ts2 and new_tsd?
plt.figure()
plt.plot(tsd1)
plt.plot(new_tsd, 'o-')
plt.plot(ts2.fillna(0), 'o')
Out:
[<matplotlib.lines.Line2D object at 0x7f87443db350>]
Question:
One important aspect of data analysis is to bring data to the same size. Pynapple provides the bin_average function to downsample data.
Question: Can you downsample tsd2 to one time point every 5 seconds?
new_tsd2 = tsd2.bin_average(5.0)
Question: Can you plot the tomato column from tsd2 as well as the downsampled version?
plt.figure()
plt.plot(tsd2['tomato'])
plt.plot(new_tsd2['tomato'], 'o-')
Out:
[<matplotlib.lines.Line2D object at 0x7f87443dba10>]
For tsd1, you want to find all the epochs for which the value is above 0.0. Pynapple provides the function threshold to get 1 dimensional time series above or below a certain value.
Question: Can you print the epochs for which tsd1 is above 0.0?
ep_above = tsd1.threshold(0.0).time_support
print(ep_above)
Out:
start end
0 11.5 12.5
1 17.5 19.5
2 24.5 27.5
3 50.5 55.5
4 58.5 59.5
5 60.5 62.5
6 63.5 64.5
...
10 83.5 84.5
11 85.5 86.5
12 87.5 88.5
13 90.5 91.5
14 92.5 93.5
15 95.5 96.5
16 97.5 98.5
shape: (17, 2), time unit: sec.
Question: can you plot tsd1 as well as the epochs for which tsd1 is above 0.0?
plt.figure()
plt.plot(tsd1)
plt.plot(tsd1.threshold(0.0), 'o-')
[plt.axvspan(s, e, alpha=0.2) for s,e in ep_above.values]
Out:
[<matplotlib.patches.Rectangle object at 0x7f8741ebef10>, <matplotlib.patches.Rectangle object at 0x7f87421c6fd0>, <matplotlib.patches.Rectangle object at 0x7f8741e5a9d0>, <matplotlib.patches.Rectangle object at 0x7f8741ec0b10>, <matplotlib.patches.Rectangle object at 0x7f8741ec15d0>, <matplotlib.patches.Rectangle object at 0x7f8741ec2090>, <matplotlib.patches.Rectangle object at 0x7f8741ec2a10>, <matplotlib.patches.Rectangle object at 0x7f8741ec3190>, <matplotlib.patches.Rectangle object at 0x7f8741ec3950>, <matplotlib.patches.Rectangle object at 0x7f8741ec3c90>, <matplotlib.patches.Rectangle object at 0x7f8741ec8850>, <matplotlib.patches.Rectangle object at 0x7f8741ec8f10>, <matplotlib.patches.Rectangle object at 0x7f8741ec9610>, <matplotlib.patches.Rectangle object at 0x7f8741ec9dd0>, <matplotlib.patches.Rectangle object at 0x7f8741eca450>, <matplotlib.patches.Rectangle object at 0x7f8741ecac90>, <matplotlib.patches.Rectangle object at 0x7f8741ec3490>]
Important
Question: Does this work? If not, please ask a TA.
import workshop_utils
path = workshop_utils.fetch_data("Mouse32-140822.nwb")
print(path)
Out:
Downloading file 'Mouse32-140822.nwb' from 'https://osf.io/jb2gd/download' to '/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/data'.
0%| | 0.00/36.6M [00:00<?, ?B/s]
0%| | 0.00/36.6M [00:00<?, ?B/s]
100%|██████████████████████████████████████| 36.6M/36.6M [00:00<00:00, 192GB/s]
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/data/Mouse32-140822.nwb
Total running time of the script: ( 0 minutes 17.533 seconds)
Download Python source code: 01_fundamentals_of_pynapple.py
Download Jupyter notebook: 01_fundamentals_of_pynapple.ipynb