reshaping module¶
Functions for reshaping arrays.
Reshape functions transform a Pandas object/NumPy array in some way.
IndexFromLike _UnionGenericAlias¶
Any object that can be coerced into a index_from argument.
to_1d_array partial¶
to_1d() with raw enabled.
to_2d_array partial¶
to_2d() with raw enabled.
to_2d_pc_array partial¶
to_2d_array with expand_axis=0.
to_2d_pr_array partial¶
to_2d_array with expand_axis=1.
align_pd_arrays function¶
align_pd_arrays(
*args,
align_index=True,
align_columns=True,
to_index=None,
to_columns=None,
axis=None,
reindex_kwargs=None
)
Align Pandas arrays against common index and/or column levels using reindexing and align_indexes() respectively.
broadcast function¶
broadcast(
*args,
to_shape=None,
align_index=None,
align_columns=None,
index_from=None,
columns_from=None,
to_frame=None,
axis=None,
to_pd=None,
keep_flex=None,
min_ndim=None,
expand_axis=None,
post_func=None,
require_kwargs=None,
reindex_kwargs=None,
merge_kwargs=None,
tile=None,
random_subset=None,
seed=None,
keep_wrap_default=None,
return_wrapper=False,
wrapper_kwargs=None,
ignore_sr_names=None,
ignore_ranges=None,
check_index_names=None,
clean_index_kwargs=None,
template_context=None
)
Bring any array-like object in args to the same shape by using NumPy-like broadcasting.
See Broadcasting.
Important
The major difference to NumPy is that one-dimensional arrays will always broadcast against the row axis!
Can broadcast Pandas objects by broadcasting their index/columns with broadcast_index().
Args
*args-
Objects to broadcast.
If the first and only argument is a mapping, will return a dict.
Allows using BCO, Ref, Default, Param, index_dict, IdxSetter, IdxSetterFactory, and templates. If an index dictionary, fills using ArrayWrapper.fill_and_set().
to_shape:tupleofint- Target shape. If set, will broadcast every object in
argstoto_shape. align_index:bool-
Whether to align index of Pandas objects using union.
Pass None to use the default.
align_columns:bool-
Whether to align columns of Pandas objects using multi-index.
Pass None to use the default.
index_from:any-
Broadcasting rule for index.
Pass None to use the default.
columns_from:any-
Broadcasting rule for columns.
Pass None to use the default.
to_frame:bool- Whether to convert all Series to DataFrames.
axis:int,sequenceormapping- See BCO.axis.
to_pd:bool,sequenceormapping-
See BCO.to_pd.
If None, converts only if there is at least one Pandas object among them.
keep_flex:bool,sequenceormapping- See BCO.keep_flex.
min_ndim:int,sequenceormapping-
See BCO.min_ndim.
If None, becomes 2 if
keep_flexis True, otherwise 1. expand_axis:int,sequenceormapping- See BCO.expand_axis.
post_func:callable,sequenceormapping-
See BCO.post_func.
Applied only when
keep_flexis False. require_kwargs:dict,sequenceormapping-
See BCO.require_kwargs.
This key will be merged with any argument-specific dict. If the mapping contains all keys in
np.require, it will be applied to all objects. reindex_kwargs:dict,sequenceormapping-
See BCO.reindex_kwargs.
This key will be merged with any argument-specific dict. If the mapping contains all keys in
pd.DataFrame.reindex, it will be applied to all objects. merge_kwargs:dict,sequenceormapping-
See BCO.merge_kwargs.
This key will be merged with any argument-specific dict. If the mapping contains all keys in
pd.DataFrame.merge, it will be applied to all objects. tile:intorindex_like- Tile the final object by the number of times or index.
random_subset:int-
Select a random subset of parameter values.
Seed can be set using NumPy before calling this function.
seed:int- Seed to make output deterministic.
keep_wrap_default:bool- Whether to keep wrapping with Default.
return_wrapper:bool- Whether to also return the wrapper associated with the operation.
wrapper_kwargs:dict- Keyword arguments passed to ArrayWrapper.
ignore_sr_names:bool- See broadcast_index().
ignore_ranges:bool- See broadcast_index().
check_index_names:bool- See broadcast_index().
clean_index_kwargs:dict- Keyword arguments passed to clean_index().
template_context:dict- Context used to substitute templates.
For defaults, see broadcasting.
Any keyword argument that can be associated with an object can be passed as
- a const that is applied to all objects,
- a sequence with value per object, and
- a mapping with value per object name and the special key
_defdenoting the default value.
Additionally, any object can be passed wrapped with BCO, which ibutes will override any of the above arguments if not None.
Usage
- Without broadcasting index and columns:
>>> from vectorbtpro import *
>>> v = 0
>>> a = np.array([1, 2, 3])
>>> sr = pd.Series([1, 2, 3], index=pd.Index(['x', 'y', 'z']), name='a')
>>> df = pd.DataFrame(
... [[1, 2, 3], [4, 5, 6], [7, 8, 9]],
... index=pd.Index(['x2', 'y2', 'z2']),
... columns=pd.Index(['a2', 'b2', 'c2']),
... )
>>> for i in vbt.broadcast(
... v, a, sr, df,
... index_from='keep',
... columns_from='keep',
... align_index=False
... ): print(i)
0 1 2
0 0 0 0
1 0 0 0
2 0 0 0
0 1 2
0 1 2 3
1 1 2 3
2 1 2 3
a a a
x 1 1 1
y 2 2 2
z 3 3 3
a2 b2 c2
x2 1 2 3
y2 4 5 6
z2 7 8 9
- Take index and columns from the argument at specific position:
>>> for i in vbt.broadcast(
... v, a, sr, df,
... index_from=2,
... columns_from=3,
... align_index=False
... ): print(i)
a2 b2 c2
x 0 0 0
y 0 0 0
z 0 0 0
a2 b2 c2
x 1 2 3
y 1 2 3
z 1 2 3
a2 b2 c2
x 1 1 1
y 2 2 2
z 3 3 3
a2 b2 c2
x 1 2 3
y 4 5 6
z 7 8 9
- Broadcast index and columns through stacking:
>>> for i in vbt.broadcast(
... v, a, sr, df,
... index_from='stack',
... columns_from='stack',
... align_index=False
... ): print(i)
a2 b2 c2
x x2 0 0 0
y y2 0 0 0
z z2 0 0 0
a2 b2 c2
x x2 1 2 3
y y2 1 2 3
z z2 1 2 3
a2 b2 c2
x x2 1 1 1
y y2 2 2 2
z z2 3 3 3
a2 b2 c2
x x2 1 2 3
y y2 4 5 6
z z2 7 8 9
- Set index and columns manually:
>>> for i in vbt.broadcast(
... v, a, sr, df,
... index_from=['a', 'b', 'c'],
... columns_from=['d', 'e', 'f'],
... align_index=False
... ): print(i)
d e f
a 0 0 0
b 0 0 0
c 0 0 0
d e f
a 1 2 3
b 1 2 3
c 1 2 3
d e f
a 1 1 1
b 2 2 2
c 3 3 3
d e f
a 1 2 3
b 4 5 6
c 7 8 9
- Pass arguments as a mapping returns a mapping:
>>> vbt.broadcast(
... dict(v=v, a=a, sr=sr, df=df),
... index_from='stack',
... align_index=False
... )
{'v': a2 b2 c2
x x2 0 0 0
y y2 0 0 0
z z2 0 0 0,
'a': a2 b2 c2
x x2 1 2 3
y y2 1 2 3
z z2 1 2 3,
'sr': a2 b2 c2
x x2 1 1 1
y y2 2 2 2
z z2 3 3 3,
'df': a2 b2 c2
x x2 1 2 3
y y2 4 5 6
z z2 7 8 9}
- Keep all results in a format suitable for flexible indexing apart from one:
>>> vbt.broadcast(
... dict(v=v, a=a, sr=sr, df=df),
... index_from='stack',
... keep_flex=dict(_def=True, df=False),
... require_kwargs=dict(df=dict(dtype=float)),
... align_index=False
... )
{'v': array([0]),
'a': array([1, 2, 3]),
'sr': array([[1],
[2],
[3]]),
'df': a2 b2 c2
x x2 1.0 2.0 3.0
y y2 4.0 5.0 6.0
z z2 7.0 8.0 9.0}
- Specify arguments per object using BCO:
>>> df_bco = vbt.BCO(df, keep_flex=False, require_kwargs=dict(dtype=float))
>>> vbt.broadcast(
... dict(v=v, a=a, sr=sr, df=df_bco),
... index_from='stack',
... keep_flex=True,
... align_index=False
... )
{'v': array([0]),
'a': array([1, 2, 3]),
'sr': array([[1],
[2],
[3]]),
'df': a2 b2 c2
x x2 1.0 2.0 3.0
y y2 4.0 5.0 6.0
z z2 7.0 8.0 9.0}
- Introduce a parameter that should build a Cartesian product of its values and other objects:
>>> df_bco = vbt.BCO(df, keep_flex=False, require_kwargs=dict(dtype=float))
>>> p_bco = vbt.BCO(pd.Param([1, 2, 3], name='my_p'))
>>> vbt.broadcast(
... dict(v=v, a=a, sr=sr, df=df_bco, p=p_bco),
... index_from='stack',
... keep_flex=True,
... align_index=False
... )
{'v': array([0]),
'a': array([1, 2, 3, 1, 2, 3, 1, 2, 3]),
'sr': array([[1],
[2],
[3]]),
'df': my_p 1 2 3
a2 b2 c2 a2 b2 c2 a2 b2 c2
x x2 1.0 2.0 3.0 1.0 2.0 3.0 1.0 2.0 3.0
y y2 4.0 5.0 6.0 4.0 5.0 6.0 4.0 5.0 6.0
z z2 7.0 8.0 9.0 7.0 8.0 9.0 7.0 8.0 9.0,
'p': array([[1, 1, 1, 2, 2, 2, 3, 3, 3],
[1, 1, 1, 2, 2, 2, 3, 3, 3],
[1, 1, 1, 2, 2, 2, 3, 3, 3]])}
- Build a Cartesian product of all parameters:
>>> vbt.broadcast(
... dict(
... a=vbt.Param([1, 2, 3]),
... b=vbt.Param(['x', 'y']),
... c=vbt.Param([False, True])
... )
... )
{'a': array([[1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3]]),
'b': array([['x', 'x', 'y', 'y', 'x', 'x', 'y', 'y', 'x', 'x', 'y', 'y']], dtype='<U1'),
'c': array([[False, True, False, True, False, True, False, True, False, True, False, True]])}
- Build a Cartesian product of two groups of parameters - (a, d) and (b, c):
>>> vbt.broadcast(
... dict(
... a=vbt.Param([1, 2, 3], level=0),
... b=vbt.Param(['x', 'y'], level=1),
... d=vbt.Param([100., 200., 300.], level=0),
... c=vbt.Param([False, True], level=1)
... )
... )
{'a': array([[1, 1, 2, 2, 3, 3]]),
'b': array([['x', 'y', 'x', 'y', 'x', 'y']], dtype='<U1'),
'd': array([[100., 100., 200., 200., 300., 300.]]),
'c': array([[False, True, False, True, False, True]])}
- Select a random subset of parameter combinations:
>>> vbt.broadcast(
... dict(
... a=vbt.Param([1, 2, 3]),
... b=vbt.Param(['x', 'y']),
... c=vbt.Param([False, True])
... ),
... random_subset=5,
... seed=42
... )
{'a': array([[1, 2, 3, 3, 3]]),
'b': array([['x', 'x', 'x', 'x', 'y']], dtype='<U1'),
'c': array([[False, True, False, True, False]])}
broadcast_array_to function¶
Broadcast an array-like object to a target shape using vectorbt's broadcasting rules.
broadcast_arrays function¶
Broadcast array-like objects using vectorbt's broadcasting rules.
Optionally to a target shape.
broadcast_combs function¶
Align an axis of each array using a combinatoric function and broadcast their indexes.
Usage
>>> from vectorbtpro import *
>>> from vectorbtpro.base.reshaping import broadcast_combs
>>> df = pd.DataFrame([[1, 2, 3], [3, 4, 5]], columns=pd.Index(['a', 'b', 'c'], name='df_param'))
>>> df2 = pd.DataFrame([[6, 7], [8, 9]], columns=pd.Index(['d', 'e'], name='df2_param'))
>>> sr = pd.Series([10, 11], name='f')
>>> new_df, new_df2, new_sr = broadcast_combs((df, df2, sr))
>>> new_df
df_param a b c
df2_param d e d e d e
0 1 1 2 2 3 3
1 3 3 4 4 5 5
>>> new_df2
df_param a b c
df2_param d e d e d e
0 6 7 6 7 6 7
1 8 9 8 9 8 9
>>> new_sr
df_param a b c
df2_param d e d e d e
0 10 10 10 10 10 10
1 11 11 11 11 11 11
broadcast_index function¶
broadcast_index(
args,
to_shape,
index_from=None,
axis=0,
ignore_sr_names=None,
ignore_ranges=None,
check_index_names=None,
**clean_index_kwargs
)
Produce a broadcast index/columns.
Args
args:iterableofarray_like- Array-like objects.
to_shape:tupleofint- Target shape.
index_from:any-
Broadcasting rule for this index/these columns.
Accepts the following values:
- 'keep' or None - keep the original index/columns of the objects in
args - 'stack' - stack different indexes/columns using stack_indexes()
- 'strict' - ensure that all Pandas objects have the same index/columns
- 'reset' - reset any index/columns (they become a simple range)
- integer - use the index/columns of the i-th object in
args - everything else will be converted to
pd.Index
- 'keep' or None - keep the original index/columns of the objects in
axis:int- Set to 0 for index and 1 for columns.
ignore_sr_names:bool-
Whether to ignore Series names if they are in conflict.
Conflicting Series names are those that are different but not None.
ignore_ranges:bool- Whether to ignore indexes of type
pd.RangeIndex. check_index_names:bool- See is_index_equal().
**clean_index_kwargs- Keyword arguments passed to clean_index().
For defaults, see broadcasting.
Note
Series names are treated as columns with a single element but without a name. If a column level without a name loses its meaning, better to convert Series to DataFrames with one column prior to broadcasting. If the name of a Series is not that important, better to drop it altogether by setting it to None.
broadcast_shapes function¶
Broadcast shape-like objects using vectorbt's broadcasting rules.
broadcast_to function¶
Broadcast arg1 to arg2.
Argument arg2 can be a shape, an instance of ArrayWrapper, or any array-like object.
Pass None to index_from/columns_from to use index/columns of the second argument.
Keyword arguments **kwargs are passed to broadcast().
Usage
>>> from vectorbtpro import *
>>> from vectorbtpro.base.reshaping import broadcast_to
>>> a = np.array([1, 2, 3])
>>> sr = pd.Series([4, 5, 6], index=pd.Index(['x', 'y', 'z']), name='a')
>>> broadcast_to(a, sr)
x 1
y 2
z 3
Name: a, dtype: int64
>>> broadcast_to(sr, a)
array([4, 5, 6])
broadcast_to_array_of function¶
Broadcast arg1 to the shape (1, *arg2.shape).
arg1 must be either a scalar, a 1-dim array, or have 1 dimension more than arg2.
Usage
>>> from vectorbtpro import *
>>> from vectorbtpro.base.reshaping import broadcast_to_array_of
>>> broadcast_to_array_of([0.1, 0.2], np.empty((2, 2)))
[[[0.1 0.1]
[0.1 0.1]]
[[0.2 0.2]
[0.2 0.2]]]
broadcast_to_axis_of function¶
Broadcast arg1 to an axis of arg2.
If arg2 has less dimensions than requested, will broadcast arg1 to a single number.
For other keyword arguments, see broadcast().
get_multiindex_series function¶
Get Series with a multi-index.
If DataFrame has been passed, must at maximum have one row or column.
index_to_frame function¶
Convert Index to DataFrame.
index_to_series function¶
Convert Index to Series.
make_symmetric function¶
Make arg symmetric.
The index and columns of the resulting DataFrame will be identical.
Requires the index and columns to have the same number of levels.
Pass sort=False if index and columns should not be sorted, but concatenated and get duplicates removed.
Usage
>>> from vectorbtpro import *
>>> from vectorbtpro.base.reshaping import make_symmetric
>>> df = pd.DataFrame([[1, 2], [3, 4]], index=['a', 'b'], columns=['c', 'd'])
>>> make_symmetric(df)
a b c d
a NaN NaN 1.0 2.0
b NaN NaN 3.0 4.0
c 1.0 3.0 NaN NaN
d 2.0 4.0 NaN NaN
mapping_to_series function¶
Convert a mapping-like object to Series.
repeat function¶
Repeat arg n times along the specified axis.
repeat_shape function¶
Repeat shape n times along the specified axis.
resolve_ref function¶
Resolve a potential reference.
soft_to_ndim function¶
Try to softly bring arg to the specified number of dimensions ndim (max 2).
tile function¶
Tile arg n times along the specified axis.
tile_shape function¶
Tile shape n times along the specified axis.
Identical to repeat_shape(). Exists purely for naming consistency.
to_1d function¶
Reshape argument to one dimension.
If raw is True, returns NumPy array. If 2-dim, will collapse along axis 1 (i.e., DataFrame with one column to Series).
to_1d_array_nb function¶
Resize array to one dimension.
to_1d_shape function¶
Convert a shape-like object to a 1-dim shape.
to_2d function¶
Reshape argument to two dimensions.
If raw is True, returns NumPy array. If 1-dim, will expand along axis 1 (i.e., Series to DataFrame with one column).
to_2d_array_nb function¶
Resize array to two dimensions.
to_2d_pc_array_nb function¶
to_2d_array_nb() with expand_axis=0.
to_2d_pr_array_nb function¶
to_2d_array_nb() with expand_axis=1.
to_2d_shape function¶
Convert a shape-like object to a 2-dim shape.
to_any_array function¶
Convert any array-like object to an array.
Pandas objects are kept as-is unless raw is True.
to_dict function¶
Convert object to dict.
to_pd_array function¶
Convert any array-like object to a Pandas object.
to_tuple_shape function¶
Convert a shape-like object to a tuple.
unstack_to_array function¶
Reshape arg based on its multi-index into a multi-dimensional array.
Use levels to specify what index levels to unstack and in which order.
Usage
>>> from vectorbtpro import *
>>> from vectorbtpro.base.reshaping import unstack_to_array
>>> index = pd.MultiIndex.from_arrays(
... [[1, 1, 2, 2], [3, 4, 3, 4], ['a', 'b', 'c', 'd']])
>>> sr = pd.Series([1, 2, 3, 4], index=index)
>>> unstack_to_array(sr).shape
(2, 2, 4)
>>> unstack_to_array(sr)
[[[ 1. nan nan nan]
[nan 2. nan nan]]
[[nan nan 3. nan]
[nan nan nan 4.]]]
>>> unstack_to_array(sr, levels=(2, 0))
[[ 1. nan]
[ 2. nan]
[nan 3.]
[nan 4.]]
unstack_to_df function¶
Reshape arg based on its multi-index into a DataFrame.
Use index_levels to specify what index levels will form new index, and column_levels for new columns. Set symmetric to True to make DataFrame symmetric.
Usage
>>> from vectorbtpro import *
>>> from vectorbtpro.base.reshaping import unstack_to_df
>>> index = pd.MultiIndex.from_arrays(
... [[1, 1, 2, 2], [3, 4, 3, 4], ['a', 'b', 'c', 'd']],
... names=['x', 'y', 'z'])
>>> sr = pd.Series([1, 2, 3, 4], index=index)
>>> unstack_to_df(sr, index_levels=(0, 1), column_levels=2)
z a b c d
x y
1 3 1.0 NaN NaN NaN
1 4 NaN 2.0 NaN NaN
2 3 NaN NaN 3.0 NaN
2 4 NaN NaN NaN 4.0
wrap_broadcasted function¶
wrap_broadcasted(
new_obj,
old_obj=None,
axis=None,
is_pd=False,
new_index=None,
new_columns=None,
ignore_ranges=None
)
If the newly brodcasted array was originally a Pandas object, make it Pandas object again and assign it the newly broadcast index/columns.
BCO class¶
Class that represents an object passed to broadcast().
If any value is None, mostly defaults to the global value passed to broadcast().
Superclasses
Inherited members
- DefineMixin.asdict()
- DefineMixin.assert_field_not_missing()
- DefineMixin.get_field()
- DefineMixin.hash
- DefineMixin.hash_key
- DefineMixin.is_field_missing()
- DefineMixin.is_field_optional()
- DefineMixin.is_field_required()
- DefineMixin.merge_over()
- DefineMixin.merge_with()
- DefineMixin.replace()
- DefineMixin.resolve()
- DefineMixin.resolve_field()
- Hashable.get_hash()
axis class variable¶
Axis to broadcast.
Set to None to broadcast all axes.
context class variable¶
Context used in evaluation of templates.
Will be merged over template_context.
expand_axis class variable¶
Axis to expand if the array is 1-dim but the target shape is 2-dim.
keep_flex class variable¶
Whether to keep the raw version of the output for flexible indexing.
Only makes sure that the array can broadcast to the target shape.
merge_kwargs class variable¶
Keyword arguments passed to column_stack_merge().
min_ndim class variable¶
Minimum number of dimensions.
post_func class variable¶
Function to post-process the output array.
reindex_kwargs class variable¶
Keyword arguments passed to pd.DataFrame.reindex.
require_kwargs class variable¶
Keyword arguments passed to np.require.
to_pd class variable¶
Whether to convert the output array to a Pandas object.
value class variable¶
Value of the object.
Default class¶
Class for wrapping default values.
Superclasses
Inherited members
- DefineMixin.asdict()
- DefineMixin.assert_field_not_missing()
- DefineMixin.get_field()
- DefineMixin.hash
- DefineMixin.hash_key
- DefineMixin.is_field_missing()
- DefineMixin.is_field_optional()
- DefineMixin.is_field_required()
- DefineMixin.merge_over()
- DefineMixin.merge_with()
- DefineMixin.replace()
- DefineMixin.resolve()
- DefineMixin.resolve_field()
- Hashable.get_hash()
value class variable¶
Default value.
Ref class¶
Class for wrapping references to other values.
Superclasses
Inherited members
- DefineMixin.asdict()
- DefineMixin.assert_field_not_missing()
- DefineMixin.get_field()
- DefineMixin.hash
- DefineMixin.hash_key
- DefineMixin.is_field_missing()
- DefineMixin.is_field_optional()
- DefineMixin.is_field_required()
- DefineMixin.merge_over()
- DefineMixin.merge_with()
- DefineMixin.replace()
- DefineMixin.resolve()
- DefineMixin.resolve_field()
- Hashable.get_hash()
key class variable¶
Reference to another key.