%matplotlib inline
Download
This notebook can be downloaded as fundamentals_of_pynapple.ipynb. See the button at the top right to download as markdown or pdf.
Learning the fundamentals of pynapple#
Learning objectives#
Instantiate the pynapple objects
Make the pynapple objects interact
Use numpy with pynapple
Slicing pynapple objects
Adding metadata to pynapple objects
Learn the core functions of pynapple
The pynapple documentation can be found here.
The documentation for objects and method of the core of pynapple is here.
Let’s start by importing the pynapple package and matplotlib to see if everything is correctly installed.
If an import fails, you can do !pip install pynapple matplotlib in a cell to fix it.
import pynapple as nap
import matplotlib.pyplot as plt
import numpy as np
import workshop_utils
WARNING:2025-02-04 19:31:17,438:jax._src.xla_bridge:987: An NVIDIA GPU may be present on this machine, but a CUDA-enabled jaxlib is not installed. Falling back to cpu.
/home/agent/workspace/rorse_ccn-software-jan-2025_main/lib/python3.11/site-packages/nemos/_documentation_utils/plotting.py:38: UserWarning: plotting functions contained within `_documentation_utils` are intended for nemos's documentation. Feel free to use them, but they will probably not work as intended with other datasets / in other contexts.
warnings.warn(
For this notebook we will work with fake data. The following cells generate a set of variables that we will use to create the different pynapple objects.
var1 = np.random.randn(100) # Variable 1
tsp1 = np.arange(100) # The timesteps of variable 1
var2 = np.random.randn(100, 3) # Variable 2
tsp2 = np.arange(0, 100, 1) # The timesteps of variable 2
col2 = ['pineapple', 'banana', 'tomato'] # The name of each columns of var2
var3 = np.random.randn(1000, 4, 5) # Variable 3
tsp3 = np.arange(0, 100, 0.1) # The timesteps of variable 3
random_times_1 = np.array([3.14, 37.0, 42.0])
random_times_2 = np.array([10, 25, 50, 70])
random_times_3 = np.sort(np.random.uniform(10, 80, 100))
starts_1 = np.array([10000, 60000, 90000]) # starts of an epoch in `ms`
ends_1 = np.array([20000, 80000, 95000]) # ends in `ms`
Instantiate pynapple objects#
This is a lot of variables to carry around. pynapple can help reduce the size of the workspace. Here we will instantiate all the different pynapple objects with the variables created above.
Let’s start with the simple ones.
Question: Can you instantiate the right pynapple objects for var1, var2 and var3? Objects should be named respectively tsd1, tsd2 and tsd3. Don’t forget the column name for var2.
tsd1 = nap.Tsd(t=tsp1, d=var1)
tsd2 = nap.TsdFrame(t=tsp2, d=var2, columns = col2)
tsd3 = nap.TsdTensor(t=tsp3, d=var3)
Question: Can you print tsd1?
print(tsd1)
Time (s)
---------- ----------
0.0 0.414295
1.0 -0.498169
2.0 0.612998
3.0 -0.387052
4.0 0.0192042
5.0 -0.118413
6.0 0.470946
...
93.0 0.645261
94.0 -0.815741
95.0 -1.10668
96.0 0.312556
97.0 -0.861115
98.0 -0.561451
99.0 0.800876
dtype: float64, shape: (100,)
Question: Can you print tsd2?
print(tsd2)
Time (s) pineapple banana tomato
---------- ----------- -------- --------
0.0 2.15885 -0.15103 0.28947
1.0 -3.45377 0.04976 1.39076
2.0 -0.42374 -0.16957 -0.40881
3.0 -0.38455 1.79036 0.59326
4.0 0.73103 -0.70372 1.48036
5.0 -0.04752 -0.24763 -0.23105
6.0 0.23251 -0.96934 -0.48325
... ... ... ...
93.0 0.51631 -0.5443 0.35771
94.0 -0.40864 0.56009 1.7266
95.0 0.15226 -2.40364 -0.39512
96.0 -0.29181 2.00801 0.56663
97.0 -0.45344 -1.04243 0.89887
98.0 -0.05768 1.15317 0.08583
99.0 1.03635 2.82941 0.92098
dtype: float64, shape: (100, 3)
Question: Can you print tsd3?
print(tsd3)
Time (s)
---------- -------------------------------
0.0 [[-0.27444 ... -0.242992] ...]
0.1 [[-2.053107 ... -1.819694] ...]
0.2 [[1.56034 ... 1.165979] ...]
0.3 [[-0.259274 ... 0.944568] ...]
0.4 [[-1.098585 ... -0.612027] ...]
0.5 [[-0.584139 ... 2.774802] ...]
0.6 [[ 1.61469 ... -0.922198] ...]
...
99.3 [[-0.505612 ... -0.587023] ...]
99.4 [[-0.290228 ... -0.349736] ...]
99.5 [[0.824908 ... 1.9689 ] ...]
99.6 [[ 1.32366 ... -1.084538] ...]
99.7 [[0.4267 ... 1.179362] ...]
99.8 [[-1.205506 ... -1.352063] ...]
99.9 [[ 0.656868 ... -0.046613] ...]
dtype: float64, shape: (1000, 4, 5)
Question: Can you create an IntervalSet called ep out of starts_1 and ends_1 and print it? Be careful, times given above are in ms.
ep = nap.IntervalSet(start=starts_1, end=ends_1, time_units='ms')
print(ep)
index start end
0 10 20
1 60 80
2 90 95
shape: (3, 2), time unit: sec.
The experiment generated a set of timestamps from 3 different channels.
Question: Can you instantiate the corresponding pynapple object (ts1, ts2, ts3) for each one of them?
ts1 = nap.Ts(t=random_times_1)
ts2 = nap.Ts(t=random_times_2)
ts3 = nap.Ts(t=random_times_3)
This is a lot of timestamps to carry around as well.
Question: Can you instantiate the right pynapple object (call it tsgroup) to group them together?
tsgroup = nap.TsGroup({0:ts1, 1:ts2, 2:ts3})
Question: … and print it?
print(tsgroup)
Index rate
------- -------
0 0.03918
1 0.05224
2 1.30598
Interaction between pynapple objects#
We reduced 12 variables in our workspace to 5 using pynapple. Now we can see how the objects interact.
Question: Can you print the time_support of tsgroup?
print(tsgroup.time_support)
index start end
0 3.14 79.7107
shape: (1, 2), time unit: sec.
The experiment ran from 0 to 100 seconds and as you can see, the TsGroup object shows the rate. But the rate is not accurate as it was computed over the default time_support.
Question: can you recreate the tsgroup object passing the right time_support during initialisation?
tsgroup = nap.TsGroup({0:ts1, 1:tsd2, 2:ts3}, time_support = nap.IntervalSet(0, 100))
Question: Can you print the time_support and rate to see how they changed?
print(tsgroup.time_support)
print(tsgroup.rate)
index start end
0 0 100
shape: (1, 2), time unit: sec.
0 0.03
1 1.00
2 1.00
Name: rate, dtype: float64
Now you realized the variable tsd1 has some noise. The good signal is between 10 and 30 seconds and 50 and 100.
Question: Can you create an IntervalSet object called ep_signal and use it to restrict the variable tsd1?
ep_signal = nap.IntervalSet(start=[10, 50], end=[30, 100])
tsd1 = tsd1.restrict(ep_signal)
print(tsd1)
print(tsd1.time_support)
Time (s)
---------- ----------
10.0 -2.2004
11.0 0.511708
12.0 0.0792929
13.0 1.81573
14.0 -2.32427
15.0 0.134434
16.0 1.00891
...
93.0 0.645261
94.0 -0.815741
95.0 -1.10668
96.0 0.312556
97.0 -0.861115
98.0 -0.561451
99.0 0.800876
dtype: float64, shape: (71,)
index start end
0 10 30
1 50 100
shape: (2, 2), time unit: sec.
ep_tmp = nap.IntervalSet(np.sort(np.random.uniform(0, 100, 20)))
print(ep_tmp)
index start end
0 6.1913 6.75218
1 8.37167 14.9699
2 16.5965 19.4499
3 22.8951 24.2491
4 32.9202 41.3465
5 44.3758 52.4088
6 70.8312 79.492
7 80.3772 80.6149
8 84.4306 90.7292
9 92.14 98.9179
shape: (10, 2), time unit: sec.
Question: Can you do the intersection of ep_signal and ep_tmp?
print(ep_signal.intersect(ep_tmp))
index start end
0 10 14.9699
1 16.5965 19.4499
2 22.8951 24.2491
3 50 52.4088
4 70.8312 79.492
5 80.3772 80.6149
6 84.4306 90.7292
7 92.14 98.9179
shape: (8, 2), time unit: sec.
workshop_utils.visualize_intervals([ep_signal, ep_tmp, ep_signal.intersect(ep_tmp)])
Question: Can you do the union of ep_signal and ep_tmp?
print(ep_signal.union(ep_tmp))
index start end
0 6.1913 6.75218
1 8.37167 30
2 32.9202 41.3465
3 44.3758 100
shape: (4, 2), time unit: sec.
Question: … and visualize it?
workshop_utils.visualize_intervals([ep_signal, ep_tmp, ep_signal.union(ep_tmp)])
Question: Can you do the difference of ep_signal and ep_tmp?
print(ep_signal.set_diff(ep_tmp))
index start end
0 14.9699 16.5965
1 19.4499 22.8951
2 24.2491 30
3 52.4088 70.8312
4 79.492 80.3772
5 80.6149 84.4306
6 90.7292 92.14
7 98.9179 100
shape: (8, 2), time unit: sec.
Question: … and visualize it?
workshop_utils.visualize_intervals([ep_signal, ep_tmp, ep_signal.set_diff(ep_tmp)])
Numpy & pynapple#
Pynapple objects behaves very similarly like numpy array. They can be sliced with the following syntax :
tsd[0:10] # First 10 elements
Arithmetical operations are available as well :
tsd = tsd + 1
Finally numpy functions works directly. Let’s imagine tsd3 is a movie with frame size (4,5).
Question: Can you compute the average frame along the time axis using np.mean and print the result?
print(np.mean(tsd3, 0))
[[-0.01834914 -0.0614865 -0.0048229 -0.01447412 0.00099555]
[ 0.02441297 0.0793515 -0.03486014 -0.02372436 -0.00060716]
[-0.07160526 0.03830993 0.03153605 -0.03500346 0.06647013]
[ 0.00748903 -0.03747333 -0.00241568 0.00525445 0.01553654]]
Question:: can you compute the average of tsd2 for each timestamps and print it?
print(np.mean(tsd2, 1))
Time (s)
---------- ---------
0.0 0.765761
1.0 -0.671084
2.0 -0.334039
3.0 0.666356
4.0 0.502559
5.0 -0.175404
6.0 -0.406694
...
93.0 0.109905
94.0 0.626014
95.0 -0.882166
96.0 0.760946
97.0 -0.199
98.0 0.393773
99.0 1.59558
dtype: float64, shape: (100,)
Notice how the output in the second case is still a pynapple object. In most cases, applying a numpy function will return a pynapple object if the time index is preserved.
Slicing pynapple objects#
Multiple methods exists to slice pynapple object. This parts reviews them.
IntervalSet also behaves like numpy array.
Question: Can you extract the first and last epoch of ep in a new IntervalSet?
print(ep[[0,2]])
index start end
0 10 20
1 90 95
shape: (2, 2), time unit: sec.
Sometimes you want to get a data point as close as possible in time to another timestamps.
Question: Using the get method, can you get the data point from tsd3 as close as possible to the time 50.1 seconds?
print(tsd3.get(50.1))
[[-2.58293688 0.5770245 -1.05523772 -0.31062255 -0.79834456]
[-0.70037823 0.66168131 -0.28896602 -0.89508804 0.69986547]
[ 1.8913538 -0.74794689 -0.09183473 -0.37394185 0.28022383]
[-0.34678062 -0.30930667 1.140357 0.09151186 -0.30037648]]
Metadata#
Metadata are ubiquitous in neuroscience. They can be added to 3 pynapple objects :
TsGroup: to label neurons in electrophysiologyIntervalSet: to label intervalsTsdFrame: to label neurons in calcium imaging
Question: Can you run the following command tsgroup['planet'] = ['mars', 'venus', 'saturn']
tsgroup['planet'] = ['mars', 'venus', 'saturn']
Question: … and print it?
print(tsgroup)
Index rate planet
------- ------ --------
0 0.03 mars
1 1 venus
2 1 saturn
The object ep has 3 epochs labelled ['left', 'right', 'left'].
Question: Can you add them as a metadata column called direction?
ep['direction'] = ['left', 'right', 'left']
print(ep)
index start end direction
0 10 20 left
1 60 80 right
2 90 95 left
shape: (3, 2), time unit: sec.
The object tsd2 has 3 columns. Each column correspond to the rgb colors [(0,0,1), (0.5, 0.5, 1), (0.1, 0.2, 0.3)].
Question: Can you add them as metadata of tsd2?
tsd2['colors'] = [(0,0,1), (0.5, 0.5, 1), (0.1, 0.2, 0.3)]
print(tsd2)
Time (s) pineapple banana tomato
---------- ----------- ------------- ---------------
0.0 2.15885 -0.15103 0.28947
1.0 -3.45377 0.04976 1.39076
2.0 -0.42374 -0.16957 -0.40881
3.0 -0.38455 1.79036 0.59326
4.0 0.73103 -0.70372 1.48036
5.0 -0.04752 -0.24763 -0.23105
6.0 0.23251 -0.96934 -0.48325
... ... ... ...
93.0 0.51631 -0.5443 0.35771
94.0 -0.40864 0.56009 1.7266
95.0 0.15226 -2.40364 -0.39512
96.0 -0.29181 2.00801 0.56663
97.0 -0.45344 -1.04243 0.89887
98.0 -0.05768 1.15317 0.08583
99.0 1.03635 2.82941 0.92098
Metadata
-------- -------- -------- --------
colors (0, 0, 1) (0.5, 0.5, 1) (0.1, 0.2, 0.3)
dtype: float64, shape: (100, 3)
You can also add metadata at initialization as a dictionnary using the keyword argument metadata :
tsgroup = nap.TsGroup({0:ts1, 1:ts2, 2:ts3}, metadata={'planet':['mars','venus', 'saturn']})
print(tsgroup)
Index rate planet
------- ------- --------
0 0.03918 mars
1 0.05224 venus
2 1.30598 saturn
Metadata are accessible either as attributes (i.e. tsgroup.planet) or as dictionnary-like keys (i.e. ep['direction']).
They can be used to slice objects.
Question: Can you select only the elements of tsgroup with rate below 1Hz?
print(tsgroup[tsgroup.rate<1.0])
print(tsgroup[tsgroup['rate']<1.0])
print(tsgroup.getby_threshold("rate", 1, "<"))
Index rate planet
------- ------- --------
0 0.03918 mars
1 0.05224 venus
Index rate planet
------- ------- --------
0 0.03918 mars
1 0.05224 venus
Index rate planet
------- ------- --------
0 0.03918 mars
1 0.05224 venus
Question: Can you select the intervals in ep labelled as 'left'?
print(ep[ep.direction=='left'])
index start end direction
0 10 20 left
1 90 95 left
shape: (2, 2), time unit: sec.
Special case of slicing : TsdFrame#
tsdframe = nap.TsdFrame(t=np.arange(4), d=np.random.randn(4,3),
columns = [12, 0, 1], metadata={'alpha':[2,1,0]})
print(tsdframe)
Time (s) 12 0 1
---------- -------- -------- --------
0.0 0.94358 1.4343 -1.32661
1.0 1.32038 0.25369 0.74346
2.0 -0.92539 0.50768 -2.78253
3.0 -0.34531 -0.72982 0.05274
Metadata
-------- -------- -------- --------
alpha 2 1 0
dtype: float64, shape: (4, 3)
Question: What happen when you do tsdframe[0] vs tsdframe[:,0] vs tsdframe[[12,1]]
print(tsdframe[0])
[ 0.94357806 1.43430298 -1.3266078 ]
Question: What happen when you do tsdframe.loc[0] and tsdframe.loc[[0,1]]
print(tsdframe.loc[0])
print(tsdframe.loc[[0,1]])
Time (s)
---------- ---------
0 1.4343
1 0.253687
2 0.50768
3 -0.729825
dtype: float64, shape: (4,)
Time (s) 0 1
---------- -------- --------
0.0 1.4343 -1.32661
1.0 0.25369 0.74346
2.0 0.50768 -2.78253
3.0 -0.72982 0.05274
Metadata
-------- -------- --------
alpha 1 0
dtype: float64, shape: (4, 2)
Question: What happen when you do tsdframe[tsdframe.alpha==2]
print(tsdframe[tsdframe.alpha==2])
Time (s) 12
---------- --------
0.0 0.94358
1.0 1.32038
2.0 -0.92539
3.0 -0.34531
Metadata
-------- --------
alpha 2
dtype: float64, shape: (4, 1)
Core functions of pynapple#
This part focuses on the most important core functions of pynapple.
Question: Using the count function, can you count the number of events within 1 second bins for tsgroup over the ep_signal intervals?
count = tsgroup.count(1, ep_signal)
print(count)
Time (s) 0 1 2
---------- --- --- ---
10.5 0.0 1.0 2.0
11.5 0.0 0.0 1.0
12.5 0.0 0.0 0.0
13.5 0.0 0.0 0.0
14.5 0.0 0.0 2.0
15.5 0.0 0.0 1.0
16.5 0.0 0.0 0.0
... ... ... ...
93.5 0.0 0.0 0.0
94.5 0.0 0.0 0.0
95.5 0.0 0.0 0.0
96.5 0.0 0.0 0.0
97.5 0.0 0.0 0.0
98.5 0.0 0.0 0.0
99.5 0.0 0.0 0.0
dtype: float64, shape: (70, 3)
Pynapple works directly with matplotlib. Passing a time series object to plt.plot will display the figure with the correct time axis.
Question: In two subplots, can you show the count and events over time?
plt.figure()
ax = plt.subplot(211)
plt.plot(count, 'o-')
plt.subplot(212, sharex=ax)
plt.plot(tsgroup.restrict(ep_signal).to_tsd(), '|')
[<matplotlib.lines.Line2D at 0x7f66285ef810>]
From a set of timestamps, you want to assign them a set of values with the closest point in time of another time series.
Question: Using the function value_from, can you assign values to ts2 from the tsd1 time series and call the output new_tsd?
new_tsd = ts2.value_from(tsd1)
Question: Can you plot together tsd1, ts2 and new_tsd?
plt.figure()
plt.plot(tsd1)
plt.plot(new_tsd, 'o-')
plt.plot(ts2.fillna(0), 'o')
[<matplotlib.lines.Line2D at 0x7f66283b2950>]
One important aspect of data analysis is to bring data to the same size. Pynapple provides the bin_average function to downsample data.
Question: Can you downsample tsd2 to one time point every 5 seconds?
new_tsd2 = tsd2.bin_average(5.0)
Question: Can you plot the tomato column from tsd2 as well as the downsampled version?
plt.figure()
plt.plot(tsd2['tomato'])
plt.plot(new_tsd2['tomato'], 'o-')
[<matplotlib.lines.Line2D at 0x7f6610391810>]
For tsd1, you want to find all the epochs for which the value is above 0.0. Pynapple provides the function threshold to get 1 dimensional time series above or below a certain value.
Question: Can you print the epochs for which tsd1 is above 0.0?
ep_above = tsd1.threshold(0.0).time_support
print(ep_above)
index start end
0 10.5 13.5
1 14.5 16.5
2 18.5 19.5
3 21.5 22.5
4 26.5 30.0
5 50.0 50.5
6 51.5 53.5
... ... ...
10 10.5 13.5
11 14.5 16.5
12 18.5 19.5
13 21.5 22.5
14 26.5 30.0
15 50.0 50.5
16 51.5 53.5
shape: (17, 2), time unit: sec.
Question: can you plot tsd1 as well as the epochs for which tsd1 is above 0.0?
plt.figure()
plt.plot(tsd1)
plt.plot(tsd1.threshold(0.0), 'o-')
[plt.axvspan(s, e, alpha=0.2) for s,e in ep_above.values]
[<matplotlib.patches.Rectangle at 0x7f65cc78f390>,
<matplotlib.patches.Rectangle at 0x7f662836e0d0>,
<matplotlib.patches.Rectangle at 0x7f65cc7a2350>,
<matplotlib.patches.Rectangle at 0x7f65cc7a2fd0>,
<matplotlib.patches.Rectangle at 0x7f65cc7a3e10>,
<matplotlib.patches.Rectangle at 0x7f65f867a050>,
<matplotlib.patches.Rectangle at 0x7f65cc7a9690>,
<matplotlib.patches.Rectangle at 0x7f65cc7aa210>,
<matplotlib.patches.Rectangle at 0x7f65cc7aad10>,
<matplotlib.patches.Rectangle at 0x7f65cc7ab790>,
<matplotlib.patches.Rectangle at 0x7f65cc7abd50>,
<matplotlib.patches.Rectangle at 0x7f65cc7b0b90>,
<matplotlib.patches.Rectangle at 0x7f65cc7b1590>,
<matplotlib.patches.Rectangle at 0x7f65cc7b2050>,
<matplotlib.patches.Rectangle at 0x7f65cc7b29d0>,
<matplotlib.patches.Rectangle at 0x7f65cc7b3350>,
<matplotlib.patches.Rectangle at 0x7f65cc7b3e50>]
Important#
Question: Does this work? If not, please ask a TA.
import workshop_utils
path = workshop_utils.fetch_data("Mouse32-140822.nwb")
print(path)
Downloading file 'Mouse32-140822.nwb' from 'https://osf.io/jb2gd/download' to '/home/agent/workspace/rorse_ccn-software-jan-2025_main/lib/python3.11/data'.
/home/agent/workspace/rorse_ccn-software-jan-2025_main/lib/python3.11/data/Mouse32-140822.nwb