Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
2cef000
coalesce data_types into data_type_lookup
jinimukh Dec 23, 2020
cad3e84
Merge branch 'master' of https://github.com/lux-org/lux
jinimukh Dec 27, 2020
f197884
merge
jinimukh Jan 8, 2021
1ed9655
merge fixed
jinimukh Jan 8, 2021
c56f79d
merge conflicts
jinimukh Jan 8, 2021
c0388df
merged
jinimukh Jan 8, 2021
cf045de
Merge branch 'master' of https://github.com/jinimukh/lux
jinimukh Jan 8, 2021
6ae9767
Merge branch 'master' into foo
jinimukh Jan 8, 2021
0db6376
merge upstream
jinimukh Jan 9, 2021
1e6f572
first commit
jinimukh Jan 15, 2021
7836abf
conflicts
jinimukh Feb 20, 2021
9c17abb
requirements.txt updated for pandas 1.2.2
jinimukh Feb 21, 2021
b3509f6
Merge branch 'master' of https://github.com/jinimukh/lux
jinimukh Feb 21, 2021
d428289
Merge branch 'master' of https://github.com/lux-org/lux
jinimukh Mar 12, 2021
9716088
Merge branch 'master' of https://github.com/lux-org/lux
jinimukh Mar 23, 2021
912f3cc
Merge branch 'master' of https://github.com/lux-org/lux
jinimukh Mar 30, 2021
8041f84
Merge branch 'master' of https://github.com/lux-org/lux
jinimukh Apr 8, 2021
9548b8a
config sorters implemented; tests passing when run together
jinimukh Apr 9, 2021
e5f5aa4
should fix failing interestingness tests
jinimukh Apr 9, 2021
0aa92a2
black
jinimukh Apr 9, 2021
0b8ee08
Merge branch 'master' of https://github.com/lux-org/lux into features…
jinimukh Aug 11, 2021
31ec5a6
structural changes, some refactoring
jinimukh Aug 11, 2021
5aa2178
black
jinimukh Aug 11, 2021
bd6d4b4
bug
jinimukh Aug 11, 2021
79e4a41
add action specific sorting
jinimukh Aug 12, 2021
f9e34bc
tests working
jinimukh Sep 22, 2021
ecdd3ea
documentation
jinimukh Sep 22, 2021
0141fa1
Merge branch 'master' into features/sort_interface
jinimukh Sep 22, 2021
d08c17c
fix bug
jinimukh Sep 22, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ This website contains pages that overview of the basic and advanced functionalit
source/advanced/architecture
source/advanced/executor
source/advanced/interestingness
source/advanced/sorting



Expand Down
67 changes: 67 additions & 0 deletions doc/source/advanced/sorting.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
***********************************
Ordering Outputs
***********************************

By default, Lux is trying to maximize the `interestingness function <interestingness.html>`_.
However, in some cases, perhaps you would like to sort by a different feature or attribute.
Here are some default sorters available to use:

Supported Orderings
====================


1. `"interestingness"`: This is the default sorter selected in Lux. It scores by the `interestingness function <interestingness.html>`_.

2. `"alphabetical_by_title"`: This sorter allows for alphabetical sorting based on the title.

3. `"alphabetical_by_x"` : This sorter allows for alphabetical sorting based on the attribute featured on the x-axis.

4. `"alphabetical_by_y"` : This sorter allows for alphabetical sorting based on the attribute featured on the y-axis.

Custom Orderings
====================

You can also add a custom sorter by creating a function that takes in a collection and a direction (ascending or descending)
and returns the sorted collection.

For example,

.. code-block:: python

def sort_by_multiple(collection, desc):
collection.sort(key=lambda x: (x.get_attr_by_channel("x")[0].attribute, x.get_attr_by_channel("y")[0].attribute), reverse=False)
lux.config.ordering = sort_by_multiple

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We didn't explain how lux.config.ordering can be used in the default case.

In this example, instead of sorting by just the x attribute or y attribute, we'd like to sort first by the x attribute and then sort by the y attribute.
The last line sets this to the globally defined ordering, which means that all actions will be ordered using this function.

Action-Dependent Orderings
==========================
There are some cases where we'd like to sort the outputs of different actions differently.
To do so, you can add to the :code:`lux.config.ordering_actions` dictionary. To set an action's ordering,
you can do the following, for example:

.. code-block:: python

lux.config.ordering_actions["correlation"] = "alphabetical_by_x"

To remove the ordering for the action, simply reset the dictionary's entry to an empty string, like so:

.. code-block:: python

lux.config.ordering_actions["correlation"] = ""

Changing the sorting direction
==============================

By default, Lux is trying to `maximize` an objective function, whether that be `interestingness` or some other custom function you define.
Thus, the default sorting direction is :code:`"descending"`. In order to toggle it, you can use:

.. code-block:: python

lux.config.sort = "ascending" # or "descending" or "None" for no sorting order

This is a globally defined sort order.



2 changes: 2 additions & 0 deletions doc/source/reference/gen/lux._config.config.Config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@
~Config.default_display
~Config.heatmap
~Config.interestingness_fallback
~Config.ordering
~Config.ordering_actions
~Config.label_len
~Config.number_of_bars
~Config.pandas_fallback
Expand Down
34 changes: 30 additions & 4 deletions lux/_config/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
from typing import Any, Callable, Dict, Iterable, List, Optional, Union
import lux
import warnings
from lux.utils.orderings import Ordering, OrderingDict, resolve_value
from lux.utils.tracing_utils import LuxTracer
import os
from lux._config.template import postgres_template, mysql_template
Expand All @@ -31,9 +32,12 @@ def __init__(self):
self._plotting_backend = "vegalite"
self._plotting_scale = 1
self._topk = 15
self._sort = True
self._ordering = Ordering.interestingness
self._ordering_actions = OrderingDict({})
self._number_of_bars = 10 # max no of bars displayed (rest shown as "+ k more")
self._label_len = 25 # max length of x and y axis labels
self._sort = "descending"
self._sort = True
self._pandas_fallback = True
self._interestingness_fallback = True
self.heatmap_bin_size = 40
Expand Down Expand Up @@ -127,8 +131,6 @@ def sort(self):
@sort.setter
def sort(self, flag: Union[str]):
"""
Setting parameter to determine sort order of each action

Parameters
----------
flag : Union[str]
Expand All @@ -137,13 +139,37 @@ def sort(self, flag: Union[str]):
"""
flag = flag.lower()
if isinstance(flag, str) and flag in ["none", "ascending", "descending"]:
self._sort = flag
if flag == "none":
self._sort = None
elif flag == "ascending":
self._sort = False
else:
self._sort = True
else:
warnings.warn(
"Parameter to lux.config.sort must be one of the following: 'none', 'ascending', or 'descending'.",
stacklevel=2,
)

@property
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if sort makes sense as a global option, since it is more common to do one type of sorting for an action, but a different sort mechanism for another action.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added two implementations -- one global ordering (this could be useful for sorting by attribute name or something really general that you want to standardize across all actions) and an action-wise ordering dictionary. If no ordering is defined for the specific action, then we fallback to the global ordering.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds great, let's add some docs that explain the two different types of ways to specify sorting!

def ordering(self):
return self._ordering

@ordering.setter
def ordering(self, value):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we rename all ordering as ranking? We should avoid the word sorter in places where it might be unclear what it is referring it.

"""
Parameters
----------
value : Union[str, Callable]
"interestingness", “alphabetical_by_title”, “alphabetical_by_x”, “alphabetical_by_y” , or Callable
Default available sorters or custom sorter
"""
self._ordering = resolve_value(value)

@property
def ordering_actions(self):
return self._ordering_actions

@property
def pandas_fallback(self):
return self._pandas_fallback
Expand Down
2 changes: 1 addition & 1 deletion lux/action/correlation.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ def correlation(ldf: LuxDataFrame, ignore_transpose: bool = True):
lux.Clause("?", data_model="measure"),
]
intent.extend(filter_specs)
vlist = VisList(intent, ldf)
vlist = VisList(intent, ldf, action="correlation")
examples = ""
if len(vlist) > 1:
measures = vlist[0].get_attr_by_data_model("measure")
Expand Down
2 changes: 1 addition & 1 deletion lux/action/enhance.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ def enhance(ldf):
clause.channel = ""
intent = filters + attr_specs
intent.append("?")
vlist = lux.vis.VisList.VisList(intent, ldf)
vlist = lux.vis.VisList.VisList(intent, ldf, action="Enhance")

# Then use the data populated in the vis list to compute score
for vis in vlist:
Expand Down
4 changes: 2 additions & 2 deletions lux/action/filter.py
Original file line number Diff line number Diff line change
Expand Up @@ -132,8 +132,8 @@ def get_complementary_ops(fltr_op):
# array of possible values for attribute
arr = ldf[last.attribute].unique().tolist()
output.append(lux.Clause(last.attribute, last.attribute, arr))
vlist = lux.vis.VisList.VisList(output, ldf)
vlist_copy = lux.vis.VisList.VisList(output, ldf)
vlist = lux.vis.VisList.VisList(output, ldf, action="Filter")
vlist_copy = lux.vis.VisList.VisList(output, ldf, action="Filter")
for i in range(len(vlist_copy)):
vlist[i].score = interestingness(vlist_copy[i], ldf)
vlist.sort()
Expand Down
4 changes: 2 additions & 2 deletions lux/action/temporal.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,11 @@ def temporal(ldf):
if len(vlist) == 0:
intent = [lux.Clause("?", data_type="temporal")]
intent.extend(utils.get_filter_specs(ldf._intent))
vlist = VisList(intent, ldf)
vlist = VisList(intent, ldf, action="temporal")
for vis in vlist:
vis.score = interestingness(vis, ldf)
else:
vlist = VisList(vlist)
vlist = VisList(vlist, action="temporal")
recommendation["long_description"] += (
" Lux displays the overall temporal trend first,"
+ " followed by trends across other timescales (e.g., year, month, week, day)."
Expand Down
2 changes: 1 addition & 1 deletion lux/action/univariate.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ def univariate(ldf, *args):
if ignore_rec_flag:
recommendation["collection"] = []
return recommendation
vlist = VisList(intent, ldf)
vlist = VisList(intent, ldf, recommendation["action"].lower())
for vis in vlist:
vis.score = interestingness(vis, ldf)
vlist.sort()
Expand Down
76 changes: 76 additions & 0 deletions lux/utils/orderings.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Copyright 2019-2020 The Lux Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import collections
import warnings

import lux


class Ordering:
@staticmethod
def interestingness(collection, desc):
collection.sort(key=lambda x: x.score, reverse=desc)

@staticmethod
def title(collection, desc):
collection.sort(key=lambda x: x.title, reverse=desc)

@staticmethod
def x_alpha(collection, desc):
collection.sort(key=lambda x: x.get_attr_by_channel("x")[0].attribute, reverse=desc)

@staticmethod
def y_alpha(collection, desc):
collection.sort(key=lambda x: x.get_attr_by_channel("y")[0].attribute, reverse=desc)


def resolve_value(value):
if type(value) is str:
if value == "interestingness":
return Ordering.interestingness
elif value == "alphabetical_by_title":
return Ordering.title
elif value == "alphabetical_by_x":
return Ordering.x_alpha
elif value == "alphabetical_by_y":
return Ordering.y_alpha
else:
assert callable(value), "You must pass in a default string or a custom function."
return value


class OrderingDict(collections.MutableMapping, dict):
def __getitem__(self, key):
return dict.__getitem__(self, key)

def __setitem__(self, key, value):
if key in lux.config.actions or key == "global":
dict.__setitem__(self, key, resolve_value(value))
else:
warnings.warn(
f"Key is not a valid action; must be one of the following: {self.actions.keys()}.",
stacklevel=2,
)

def __delitem__(self, key):
dict.__delitem__(self, key)

def __iter__(self):
return dict.__iter__(self)

def __len__(self):
return dict.__len__(self)

def __contains__(self, x):
return dict.__contains__(self, x)
20 changes: 10 additions & 10 deletions lux/vis/VisList.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,11 @@
class VisList:
"""VisList is a list of Vis objects."""

def __init__(self, input_lst: Union[List[Vis], List[Clause]], source=None):
def __init__(self, input_lst: Union[List[Vis], List[Clause]], source=None, action=None):
# Overloaded Constructor
self._source = source
self._input_lst = input_lst
self._action = action
if len(input_lst) > 0:
if self._is_vis_input():
self._collection = input_lst
Expand Down Expand Up @@ -229,18 +230,17 @@ def get_field(d_obj):
def set(self, field_name, field_val):
return NotImplemented

def sort(self, remove_invalid=True, descending=True):
def sort(self, remove_invalid=True):
# remove the items that have invalid (-1) score
if remove_invalid:
self._collection = list(filter(lambda x: x.score != -1, self._collection))
if lux.config.sort == "none":
return
elif lux.config.sort == "ascending":
descending = False
elif lux.config.sort == "descending":
descending = True
# sort in-place by “score” by default if available, otherwise user-specified field to sort by
self._collection.sort(key=lambda x: x.score, reverse=descending)
if lux.config.sort is not None:
if self._action is None or lux.config.ordering_actions.get(self._action) is None:
ordering_function = lux.config.ordering
else:
ordering_function = lux.config.ordering_actions.get(self._action)
# sort in-place by “score” by default if available, otherwise user-specified field to sort by
ordering_function(self._collection, lux.config.sort)

def showK(self):
k = lux.config.topk
Expand Down
Loading