Returning text & images to editor as well as audio

[drscotthawley]

Dear Hugo et al,

I have a gradio app that reads some audio and produces text and analysis graphs of it. (It doesn't actually change the audio.)
It was looking like I could use PyHARP v0.1.0 to integrate my gradio app with a DAW, and just return the original audio unaltered so it didn't affect the DAW, as well as return my graph and text.
build_endpoint took four arguments (inputs, output, process_fn, card).

But now with v 0.1.1, inputs & output are together in "components". In this new framework, how can I setup an additional output to display in HARP, e.g. of type gr.Image?

Thanks!

[drscotthawley]

Ah, ok I modified the source, build_endpoint in core.py to accept an extra kwarg for optional outputs, e.g.

def build_endpoint(model_card: ModelCard, components: list, process_fn: callable, out_img=None) -> tuple:

(where out_img is already initialized as type `gr.Image(type="filepath") and then later

      outputs=[out, out_img] if out_img is not None else [out],

...the whole thing runs fine.

However, since HARP itself doesn't seem to support the extra image display, the image return by the gradio demo just gets ignored 😢
it would be cool if HARP were to support such things, but I understand that this is not most people's intended use case.

[cwitkowitz]

Hi Scott!

Sorry we missed this until now. Currently, HARP/PyHARP (v2.0.0/v0.2.0) assume models will be either audio-in/audio-out or midi-in/midi-out.

However, as part of our next steps, we plan on moving towards a more generalized I/O, and we can certainly keep image data in mind as a potential input or output.

Thanks for your interest and suggestions! Looking forward to see what you do with HARP.