error-analysis

LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

README.md

@@ -1,13 +1 @@
----
-title: Error Analysis
-emoji: 🔥
-colorFrom: green
-colorTo: red
-sdk: streamlit
-sdk_version: 1.2.0
-app_file: app.py
-pinned: false
-license: apache-2.0
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces#reference
+# error-analysis

amazon_polarity.test.parquet

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e57ae9ce39c5251e432b4a6dce31915782276b98a7751281eb66b8cff3b46b6
+size 5864011

app.py

@@ -0,0 +1,247 @@
+    ### LIBRARIES ###
+# # Data
+from matplotlib.pyplot import legend
+import numpy as np
+import pandas as pd
+import torch
+import json
+from tqdm import tqdm
+from math import floor
+from datasets import load_dataset
+from collections import defaultdict
+from transformers import AutoTokenizer
+
+# Analysis
+# from gensim.models.doc2vec import Doc2Vec
+# from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score
+# import nltk
+# nltk.download('punkt') #make sure that punkt is downloaded
+
+# App & Visualization
+import streamlit as st
+import st_aggrid
+import altair as alt
+import plotly.graph_objects as go
+from streamlit_vega_lite import altair_component
+
+# utils
+from random import sample
+# from PIL import Image
+
+
+def down_samp(embedding):
+    """Down sample a data frame for altiar visualization """
+    # total number of positive and negative sentiments in the class
+    #embedding = embedding.groupby('slice').apply(lambda x: x.sample(frac=0.3))
+    total_size = embedding.groupby(['slice','label'], as_index=False).count()
+
+    user_data = 0
+    # if 'Your Sentences' in str(total_size['slice']):
+    #     tmp = embedding.groupby(['slice'], as_index=False).count()
+    #     val = int(tmp[tmp['slice'] == "Your Sentences"]['source'])
+    #     user_data = val
+
+    max_sample = total_size.groupby('slice').max()['content']
+
+    # # down sample to meeting altair's max values
+    # # but keep the proportional representation of groups
+    down_samp = 1/(sum(max_sample.astype(float))/(1000-user_data))
+
+    max_samp = max_sample.apply(lambda x: floor(x*down_samp)).astype(int).to_dict()
+    max_samp['Your Sentences'] = user_data
+
+    # # sample down for each group in the data frame
+    embedding = embedding.groupby('slice').apply(lambda x: x.sample(n=max_samp.get(x.name))).reset_index(drop=True)
+
+    # # order the embedding
+    return(embedding)
+
+
+def data_comparison(df):
+    # set up a dropdown select bindinf
+    # input_dropdown = alt.binding_select(options=['Negative Sentiment','Positive Sentiment'])
+    selection = alt.selection_multi(fields=['slice','label'])
+    color = alt.condition(alt.datum.slice == 'high-loss', alt.value("orange"), alt.value("steelblue"))
+    # color = alt.condition(selection, 
+    #                       alt.Color('slice:Q', legend=None),
+    #                       # scale = alt.Scale(domain = pop_domain,range=color_range)),
+    #                       alt.value('lightgray'))
+    opacity = alt.condition(selection, alt.value(0.7), alt.value(0.25))
+
+    # basic chart
+    scatter = alt.Chart(df).mark_point(size=100, filled=True).encode(
+        x=alt.X('x', axis=None),
+        y=alt.Y('y', axis=None),
+        color=color,
+        shape=alt.Shape('label', scale=alt.Scale(range=['circle', 'diamond'])),
+        tooltip=['slice','content','label','pred'],
+        opacity=opacity
+    ).properties(
+        width=1500,
+        height=1000
+    ).interactive()
+
+    legend = alt.Chart(df).mark_point().encode(
+        y=alt.Y('slice:N', axis=alt.Axis(orient='right'), title=""),
+        x=alt.X("label"),
+        shape=alt.Shape('label', scale=alt.Scale(
+            range=['circle', 'diamond']), legend=None),
+        color=color
+    ).add_selection(
+        selection
+    )
+
+    layered = scatter |legend
+
+    layered = layered.configure_axis(
+        grid=False
+    ).configure_view(
+        strokeOpacity=0
+    )
+
+    return layered
+
+
+def quant_panel(embedding_df):
+    """ Quantitative Panel Layout"""
+
+    all_metrics = {}
+    # st.warning("**Data Comparison**")
+
+    # with st.expander("how to read this chart:"):
+    #     st.markdown("* each **point** is a single sentence")
+    #     st.markdown("* the **position** of each dot is determined mathematically based upon an analysis of the words in a sentence. The **closer** two points on the visualization the **more similar** the sentences are. The **further apart ** two points on the visualization the **more different** the sentences are")
+    #     st.markdown(
+    #             " * the **shape** of each point reflects whether it a positive (diamond) or negative sentiment (circle)")
+    #     st.markdown("* the **color** of each point is the ")
+    st.altair_chart(data_comparison(down_samp(embedding_df)))
+
+def frequent_tokens(data, tokenizer, loss_quantile=0.95, top_k=200, smoothing=0.005):
+    unique_tokens = []
+    tokens = []
+    for row in tqdm(data['content']):
+        tokenized = tokenizer(row,padding=True, return_tensors='pt')
+        tokens.append(tokenized['input_ids'].flatten())
+        unique_tokens.append(torch.unique(tokenized['input_ids']))
+    losses = data['loss'].astype(float)
+    high_loss = losses.quantile(loss_quantile)
+    loss_weights = (losses > high_loss)
+    loss_weights = loss_weights / loss_weights.sum()
+    token_frequencies = defaultdict(float)
+    token_frequencies_error = defaultdict(float)
+
+    weights_uniform = np.full_like(loss_weights, 1 / len(loss_weights))
+
+    num_examples = len(data)
+    for i in tqdm(range(num_examples)):
+        for token in unique_tokens[i]:
+            token_frequencies[token.item()] += weights_uniform[i]
+            token_frequencies_error[token.item()] += loss_weights[i]
+
+    token_lrs = {k: (smoothing+token_frequencies_error[k]) / (smoothing+token_frequencies[k]) for k in token_frequencies}
+    tokens_sorted = list(map(lambda x: x[0], sorted(token_lrs.items(), key=lambda x: x[1])[::-1]))
+
+    top_tokens = []
+    for i, (token) in enumerate(tokens_sorted[:top_k]):
+        top_tokens.append(['%10s' % (tokenizer.decode(token)), '%.4f' % (token_frequencies[token]), '%.4f' % (
+            token_frequencies_error[token]), '%4.2f' % (token_lrs[token])])
+    return pd.DataFrame(top_tokens, columns=['Token', 'Freq', 'Freq error slice', 'lrs'])
+
+
+@st.cache(ttl=600)
+def get_data(spotlight, emb):
+    preds = spotlight.outputs.numpy()
+    losses = spotlight.losses.numpy()
+    embeddings = pd.DataFrame(emb, columns=['x', 'y'])
+    num_examples = len(losses)
+    # dataset_labels = [dataset[i]['label'] for i in range(num_examples)]
+    return pd.concat([pd.DataFrame(np.transpose(np.vstack([dataset[:num_examples]['content'], 
+                    dataset[:num_examples]['label'], preds, losses])), columns=['content', 'label', 'pred', 'loss']), embeddings], axis=1)
+
+
+def topic_distribution(weights, smoothing=0.01):
+    topic_frequencies = defaultdict(float)
+    topic_frequencies_spotlight = defaultdict(float)
+    weights_uniform = np.full_like(weights, 1 / len(weights))
+    num_examples = len(weights)
+    for i in range(num_examples):
+        example = dataset[i]
+        category = example['title']
+        topic_frequencies[category] += weights_uniform[i]
+        topic_frequencies_spotlight[category] += weights[i]
+
+    topic_ratios = {c: (smoothing + topic_frequencies_spotlight[c]) / (
+        smoothing + topic_frequencies[c]) for c in topic_frequencies}
+
+    categories_sorted = map(lambda x: x[0], sorted(
+        topic_ratios.items(), key=lambda x: x[1], reverse=True))
+
+    topic_distr = []
+    for category in categories_sorted:
+        topic_distr.append(['%.3f' % topic_frequencies[category], '%.3f' %
+                           topic_frequencies_spotlight[category], '%.2f' % topic_ratios[category], '%s' % category])
+
+    return pd.DataFrame(topic_distr, columns=['Overall frequency', 'Error frequency', 'Ratio', 'Category'])
+    # for category in categories_sorted:
+    #    return(topic_frequencies[category], topic_frequencies_spotlight[category], topic_ratios[category], category)
+
+
+if __name__ == "__main__":
+    ### STREAMLIT APP CONGFIG ###
+    st.set_page_config(layout="wide", page_title="Error Slice Analysis")
+    lcol, rcol = st.columns([3, 2])
+    # ******* loading the mode and the data
+    dataset = st.sidebar.selectbox(
+        "Dataset",
+        ["amazon_polarity", "squad", "movielens", "waterbirds"],
+        index=0
+    )
+
+    tokenizer = AutoTokenizer.from_pretrained(
+        "distilbert-base-uncased-finetuned-sst-2-english")
+
+    model = st.sidebar.selectbox(
+        "Model",
+        ["distilbert-base-uncased-finetuned-sst-2-english",
+            "distilbert-base-uncased-finetuned-sst-2-english"],
+        index=0
+    )
+
+    loss_quantile = st.sidebar.selectbox(
+        "Loss Quantile",
+        [0.98, 0.95, 0.9, 0.8, 0.75],
+        index = 1
+    )
+    ### LOAD DATA AND SESSION VARIABLES ###
+    data_df = pd.read_parquet('amazon_polarity.test.parquet')
+    embedding_umap = data_df[['x','y']]
+    if "user_data" not in st.session_state:
+        st.session_state["user_data"] = data_df
+    if "selected_slice" not in st.session_state:
+        st.session_state["selected_slice"] = None
+    if "embedding" not in st.session_state:
+        st.session_state["embedding"] = embedding_umap
+
+    with lcol:
+        st.title('Error Slices')
+        dataframe = data_df[['content', 'label', 'pred', 'loss']].sort_values(
+            by=['loss'], ascending=False)
+        table_html = dataframe.to_html(
+            columns=['content', 'label', 'pred', 'loss'], max_rows=100)
+        # table_html = table_html.replace("<th>", '<th align="left">')  # left-align the headers
+        st.write(dataframe)
+    # st_aggrid.AgGrid(dataframe)
+    # table_html = dataframe.to_html(columns=['content', 'label', 'pred', 'loss'], max_rows=100)
+    # table_html = table_html.replace("<th>", '<th align="left">')  # left-align the headers
+    # st.write(table_html)
+    
+    with rcol:
+        st.title('Word Distribution in Error Slice')
+        commontokens = frequent_tokens(data_df, tokenizer, loss_quantile=loss_quantile)
+        st.write(commontokens)
+    data_df['loss'] = data_df['loss'].astype(float)
+    losses = data_df['loss']
+    high_loss = losses.quantile(loss_quantile)
+    data_df['slice'] = 'high-loss'
+    data_df['slice'] = data_df['slice'].where(data_df['loss'] > high_loss, 'low-loss')
+    quant_panel(data_df)

requirements.txt

@@ -0,0 +1,313 @@
+# This file may be used to create an environment using:
+# $ conda create --name <env> --file <this file>
+# platform: osx-arm64
+abseil-cpp=20210324.2=hbdafb3b_0
+aiohttp=3.8.1=py39hb18efdd_1
+aiosignal=1.2.0=pyhd8ed1ab_0
+altair=4.2.0=pyhd8ed1ab_1
+appnope=0.1.3=pyhd8ed1ab_0
+argh=0.26.2=pyh9f0ad1d_1002
+argon2-cffi=21.3.0=pyhd8ed1ab_0
+argon2-cffi-bindings=21.2.0=py39h5161555_1
+arrow-cpp=6.0.1=py39h71c7f51_5_cpu
+astor=0.8.1=pyh9f0ad1d_0
+asttokens=2.0.5=pyhd8ed1ab_0
+async-timeout=4.0.2=pyhd8ed1ab_0
+attrs=21.4.0=pyhd8ed1ab_0
+autopep8=1.6.0=pyhd3eb1b0_0
+aws-c-auth=0.6.8=h77ca94e_1
+aws-c-cal=0.5.12=hc1327b6_7
+aws-c-common=0.6.17=h3422bc3_0
+aws-c-compression=0.2.14=haaffe3e_7
+aws-c-event-stream=0.2.7=hd0ff547_32
+aws-c-http=0.6.10=h53b0524_3
+aws-c-io=0.10.14=h3e85fa9_1
+aws-c-mqtt=0.7.10=hd8b1cef_0
+aws-c-s3=0.1.29=h6db2689_0
+aws-c-sdkutils=0.1.1=haaffe3e_4
+aws-checksums=0.1.12=haaffe3e_6
+aws-crt-cpp=0.17.10=h5d9c0f4_5
+aws-sdk-cpp=1.9.160=he5b1d48_0
+backcall=0.2.0=pyh9f0ad1d_0
+backports=1.0=py_2
+backports.functools_lru_cache=1.6.4=pyhd8ed1ab_0
+base58=2.1.1=pyhd8ed1ab_0
+beautifulsoup4=4.10.0=pyha770c72_0
+blas=2.114=openblas
+blas-devel=3.9.0=14_osxarm64_openblas
+bleach=4.1.0=pyhd8ed1ab_0
+blinker=1.4=pypi_0
+blosc=1.21.0=h9f76cd9_0
+bokeh=2.4.2=py39hca03da5_0
+boto3=1.22.0=pyhd8ed1ab_0
+botocore=1.25.0=pyhd8ed1ab_0
+bottleneck=1.3.4=py39heec5a64_0
+brotli=1.0.7=hc377ac9_0
+brotlipy=0.7.0=py39hb18efdd_1004
+brunsli=0.1=hc377ac9_1
+bzip2=1.0.8=h3422bc3_4
+c-ares=1.18.1=h3422bc3_0
+c-blosc2=2.0.4=h0095615_1
+ca-certificates=2021.10.8=h4653dfc_0
+cachetools=5.0.0=pyhd8ed1ab_0
+certifi=2021.10.8=py39h2804cbe_2
+cffi=1.15.0=py39h52b1de0_0
+cfitsio=4.0.0=h99351b2_0
+charls=2.2.0=hc377ac9_0
+charset-normalizer=2.0.12=pyhd8ed1ab_0
+click=8.0.4=py39h2804cbe_0
+cloudpickle=2.0.0=pyhd3eb1b0_0
+colorama=0.4.4=pyh9f0ad1d_0
+colorcet=2.0.6=pyhd3eb1b0_0
+cryptography=36.0.2=py39hbe5e4b8_1
+cycler=0.11.0=pyhd3eb1b0_0
+cytoolz=0.11.0=py39h1a28f6b_0
+dask=2022.2.1=pyhd3eb1b0_0
+dask-core=2022.2.1=pyhd3eb1b0_0
+dataclasses=0.8=pyhc8e2a94_3
+datasets=2.0.0=py_0
+datashader=0.13.0=pyhd3eb1b0_1
+datashape=0.5.4=py39hca03da5_1
+debugpy=1.5.1=py39hfb83b0d_0
+decorator=5.1.1=pyhd8ed1ab_0
+defusedxml=0.7.1=pyhd8ed1ab_0
+dill=0.3.4=pyhd8ed1ab_0
+distributed=2022.2.1=pyhd3eb1b0_0
+entrypoints=0.4=pyhd8ed1ab_0
+executing=0.8.3=pyhd8ed1ab_0
+filelock=3.6.0=pyhd8ed1ab_0
+flit-core=3.7.1=pyhd8ed1ab_0
+fonttools=4.31.2=pypi_0
+freetype=2.11.0=h1192e45_0
+frozenlist=1.3.0=py39hb18efdd_1
+fsspec=2022.3.0=pyhd8ed1ab_0
+future=0.18.2=py39hca03da5_1
+fuzzywuzzy=0.18.0=pypi_0
+gflags=2.2.2=hc88da5d_1004
+gh=2.7.0=h75b854d_0
+giflib=5.2.1=h1a28f6b_0
+gitdb=4.0.9=pyhd8ed1ab_0
+gitpython=3.1.27=pyhd8ed1ab_0
+glog=0.5.0=h5c6a83d_0
+grpc-cpp=1.42.0=hedfbb7c_1
+heapdict=1.0.1=pyhd3eb1b0_0
+holoviews=1.14.8=pyhd3eb1b0_0
+htmlmin=0.1.12=pypi_0
+huggingface_hub=0.5.1=py_0
+idna=3.3=pyhd8ed1ab_0
+imagecodecs=2021.11.20=py39hcb02aed_1
+imagehash=4.2.1=pypi_0
+imageio=2.9.0=pyhd3eb1b0_0
+importlib-metadata=4.11.3=py39h2804cbe_1
+importlib_metadata=4.11.3=hd8ed1ab_1
+importlib_resources=5.6.0=pyhd8ed1ab_0
+ipykernel=6.12.1=py39h32adebf_0
+ipython=8.2.0=py39h2804cbe_0
+ipython-genutils=0.2.0=pypi_0
+ipython_genutils=0.2.0=py_1
+ipywidgets=7.7.0=pyhd8ed1ab_0
+jbig=2.1=h1a28f6b_0
+jedi=0.18.1=py39h2804cbe_1
+jinja2=3.1.1=pyhd8ed1ab_0
+jmespath=1.0.0=pyhd8ed1ab_0
+joblib=1.0.1=pypi_0
+jpeg=9d=h1a28f6b_0
+jsonschema=4.4.0=pyhd8ed1ab_0
+jupyter=1.0.0=pypi_0
+jupyter-console=6.4.3=pypi_0
+jupyter_client=7.2.1=pyhd8ed1ab_0
+jupyter_core=4.9.2=py39h2804cbe_0
+jupyterlab_pygments=0.1.2=pyh9f0ad1d_0
+jupyterlab_widgets=1.1.0=pyhd8ed1ab_0
+jxrlib=1.1=h1a28f6b_2
+kaleido=0.2.1=pypi_0
+kiwisolver=1.4.2=pypi_0
+krb5=1.19.3=hf9b2bbe_0
+lcms2=2.12=hba8e193_0
+lerc=3.0=hc377ac9_0
+libaec=1.0.6=hbdafb3b_0
+libblas=3.9.0=14_osxarm64_openblas
+libbrotlicommon=1.0.9=h1c322ee_7
+libbrotlidec=1.0.9=h1c322ee_7
+libbrotlienc=1.0.9=h1c322ee_7
+libcblas=3.9.0=14_osxarm64_openblas
+libcurl=7.82.0=hb0e6552_0
+libcxx=13.0.1=h6a5c8ee_0
+libdeflate=1.8=h1a28f6b_5
+libedit=3.1.20191231=hc8eb9b7_2
+libev=4.33=h642e427_1
+libevent=2.1.10=hbae9a57_4
+libffi=3.4.2=h3422bc3_5
+libgfortran=5.0.0.dev0=11_0_1_hf114ba7_23
+libgfortran5=11.0.1.dev0=hf114ba7_23
+liblapack=3.9.0=14_osxarm64_openblas
+liblapacke=3.9.0=14_osxarm64_openblas
+libllvm11=11.1.0=h93073aa_3
+libnghttp2=1.47.0=he723fca_0
+libopenblas=0.3.20=openmp_h2209c59_0
+libpng=1.6.37=hb8d0fd4_0
+libprotobuf=3.19.1=h98b2900_0
+libsodium=1.0.18=h27ca646_1
+libssh2=1.10.0=hb80f160_2
+libthrift=0.15.0=h28a9c34_1
+libtiff=4.3.0=h74060c4_2
+libutf8proc=2.7.0=h3422bc3_0
+libwebp=1.2.2=h68602c7_0
+libwebp-base=1.2.2=h1a28f6b_0
+libzlib=1.2.11=h90dfc92_1014
+libzopfli=1.0.3=hc377ac9_0
+llvm-openmp=13.0.1=h455960f_1
+llvmlite=0.38.0=py39hd599773_1
+locket=0.2.1=py39hca03da5_2
+lz4-c=1.9.3=hbdafb3b_1
+markdown=3.3.4=py39hca03da5_0
+markupsafe=2.0.1=pypi_0
+matplotlib=3.5.1=py39hca03da5_1
+matplotlib-base=3.5.1=py39hc377ac9_1
+matplotlib-inline=0.1.3=pyhd8ed1ab_0
+missingno=0.5.1=pypi_0
+mistune=0.8.4=py39h5161555_1005
+msgpack-python=1.0.2=py39h525c30c_1
+multidict=6.0.2=py39hb18efdd_1
+multimethod=1.7=pypi_0
+multipledispatch=0.6.0=py39hca03da5_0
+multiprocess=0.70.12.2=py39hb18efdd_2
+munkres=1.1.4=py_0
+nbclient=0.5.13=pyhd8ed1ab_0
+nbconvert=6.4.5=pyhd8ed1ab_2
+nbconvert-core=6.4.5=pyhd8ed1ab_2
+nbconvert-pandoc=6.4.5=pyhd8ed1ab_2
+nbformat=5.3.0=pyhd8ed1ab_0
+ncurses=6.3=hc470f4d_0
+nest-asyncio=1.5.5=pyhd8ed1ab_0
+networkx=2.7.1=pyhd3eb1b0_0
+ninja=1.10.2=py39h525c30c_3
+notebook=6.4.10=pyha770c72_0
+numba=0.55.1=py39hb1c450a_0
+numexpr=2.8.1=py39h144ceef_0
+numpy=1.21.5=py39h25ab29e_1
+numpy-base=1.21.5=py39h974a1f5_1
+openblas=0.3.20=openmp_h745f6c2_0
+openjpeg=2.4.0=h062765e_1
+openssl=1.1.1n=h90dfc92_0
+orc=1.7.1=hcb6706d_1
+packaging=21.3=pyhd8ed1ab_0
+pandas=1.4.1=py39hc377ac9_1
+pandas-profiling=3.1.0=pypi_0
+pandoc=2.12=hca03da5_0
+pandocfilters=1.5.0=pyhd8ed1ab_0
+panel=0.12.6=pyhd3eb1b0_0
+param=1.12.0=pyhd3eb1b0_0
+parquet-cpp=1.5.1=2
+parso=0.8.3=pyhd8ed1ab_0
+partd=1.2.0=pyhd3eb1b0_1
+pexpect=4.8.0=pyh9f0ad1d_2
+phik=0.12.2=pypi_0
+pickleshare=0.7.5=py_1003
+pillow=9.1.0=pypi_0
+pip=22.0.4=pyhd8ed1ab_0
+plotly=5.7.0=py_0
+progressbar=2.5=pypi_0
+prometheus_client=0.14.0=pyhd8ed1ab_0
+prompt-toolkit=3.0.29=pyha770c72_0
+protobuf=3.20.0=pypi_0
+psutil=5.9.0=py39hb18efdd_1
+ptyprocess=0.7.0=pyhd3deb0d_0
+pure_eval=0.2.2=pyhd8ed1ab_0
+pyahocorasick=1.4.4=pypi_0
+pyarrow=6.0.1=py39hd3b58d7_5_cpu
+pyasn1=0.4.8=pypi_0
+pycodestyle=2.8.0=pyhd3eb1b0_0
+pycparser=2.21=pyhd8ed1ab_0
+pyct=0.4.6=py39hca03da5_0
+pydantic=1.9.0=pypi_0
+pydeck=0.7.1=pyh6c4a22f_0
+pygments=2.11.2=pyhd8ed1ab_0
+pympler=1.0.1=pypi_0
+pynndescent=0.5.6=pyh6c4a22f_0
+pyopenssl=22.0.0=pyhd8ed1ab_0
+pyparsing=3.0.7=pyhd8ed1ab_0
+pyrsistent=0.18.1=py39hb18efdd_1
+pysocks=1.7.1=py39h2804cbe_5
+python=3.9.12=hfc7342c_1_cpython
+python-dateutil=2.8.2=pyhd8ed1ab_0
+python-dotenv=0.19.2=pypi_0
+python-fastjsonschema=2.15.3=pyhd8ed1ab_0
+python-tzdata=2022.1=pyhd8ed1ab_0
+python-xxhash=3.0.0=py39hb18efdd_0
+python_abi=3.9=1_cp39
+pytorch=1.10.2=cpu_py39h23cb94c_0
+pytz=2022.1=pyhd8ed1ab_0
+pytz-deprecation-shim=0.1.0.post0=py39h2804cbe_1
+pyviz_comms=2.0.2=pyhd3eb1b0_0
+pywavelets=1.3.0=py39h1a28f6b_0
+pyyaml=6.0=py39hb18efdd_4
+pyzmq=22.3.0=py39h7a4232c_2
+qtconsole=5.3.0=pypi_0
+qtpy=2.0.1=pypi_0
+re2=2021.11.01=hbdafb3b_0
+readline=8.1=hedafd6a_0
+regex=2022.3.15=py39hb18efdd_1
+requests=2.27.1=pyhd8ed1ab_0
+s3transfer=0.5.2=pyhd8ed1ab_0
+sacremoses=0.0.49=pyhd8ed1ab_0
+scikit-image=0.19.2=py39h9197a36_0
+scikit-learn=1.0.2=py39hef7049f_0
+scipy=1.8.0=py39h5060c3b_1
+seaborn=0.11.2=pypi_0
+semver=2.13.0=pyh9f0ad1d_0
+send2trash=1.8.0=pyhd8ed1ab_0
+setuptools=62.0.0=py39h2804cbe_0
+simplejson=3.17.6=pypi_0
+six=1.16.0=pyh6c4a22f_0
+smmap=5.0.0=pypi_0
+snappy=1.1.8=hc88da5d_3
+sortedcontainers=2.4.0=pyhd3eb1b0_0
+soupsieve=2.3.2=pypi_0
+sqlite=3.37.1=h7e3ccbd_0
+stack_data=0.2.0=pyhd8ed1ab_0
+streamlit=1.8.1=pyhd8ed1ab_0
+streamlit-aggrid=0.2.3.post2=pypi_0
+streamlit-vega-lite=0.1.0=pypi_0
+tangled-up-in-unicode=0.1.0=pypi_0
+tbb=2021.5.0=h3e96240_1
+tblib=1.7.0=pyhd3eb1b0_0
+tenacity=8.0.1=py39hca03da5_0
+tensorboard-plugin-wit=1.8.1=pypi_0
+terminado=0.13.3=py39h2804cbe_1
+testpath=0.6.0=pyhd8ed1ab_0
+threadpoolctl=3.1.0=pyh8a188c0_0
+tifffile=2021.7.2=pyhd3eb1b0_2
+tk=8.6.12=he1e0b03_0
+tokenizers=0.11.6=pypi_0
+toml=0.10.2=pyhd3eb1b0_0
+toolz=0.11.2=pyhd8ed1ab_0
+torchvision=0.2.2=py_3
+tornado=6.1=py39hb18efdd_3
+tqdm=4.64.0=pyhd8ed1ab_0
+traitlets=5.1.1=pyhd8ed1ab_0
+transformers=4.18.0=pypi_0
+typing-extensions=4.1.1=hd8ed1ab_0
+typing_extensions=4.1.1=pyha770c72_0
+tzdata=2022a=h191b570_0
+tzlocal=4.2=py39h2804cbe_0
+umap-learn=0.5.3=py39h2804cbe_0
+urllib3=1.26.9=pyhd8ed1ab_0
+validators=0.18.2=pyhd3deb0d_0
+visions=0.7.4=pypi_0
+watchdog=2.1.7=py39hb18efdd_1
+wcwidth=0.2.5=pyh9f0ad1d_2
+webencodings=0.5.1=pypi_0
+wheel=0.37.1=pyhd8ed1ab_0
+widgetsnbextension=3.6.0=py39h2804cbe_0
+xarray=0.20.1=pyhd3eb1b0_1
+xxhash=0.8.0=h27ca646_3
+xz=5.2.5=h642e427_1
+yaml=0.2.5=h3422bc3_2
+yarl=1.7.2=py39hb18efdd_2
+zeromq=4.3.4=hbdafb3b_1
+zfp=0.5.5=hc377ac9_6
+zict=2.0.0=pyhd3eb1b0_0
+zipp=3.8.0=pyhd8ed1ab_0
+zlib=1.2.11=h90dfc92_1014
+zstd=1.5.2=h861e0a7_0