The Parrot Sketch ----------------- How to turn your computer into a parrot using only freely available tools ======================================================================== This document sketches how you can set up components like a speech recogniser, a text-to-speech system, and a dialogue management component, to realise a "parrot": a system that repeats to you what you said to it (well, what it heard anyway). Why should you want to do that? Well, besides the reasons people who own real parrots cite ("helps me against loneliness"), once you have everything set up, you can use this as the starting point for developing more advance dialogue systems. Disclaimer: Almost none of the software described here is written by us. We only provide a comprehensive installation instruction, to save you hours of scouring the net for components and fiddling around with config files to bring everything together. Many thanks to the developers of those packages! Please see the original files for the licensing conditions. 0. Prerequisites ---------------- In this step you will assemble all the necessary ingredients. This will take a while, depending on the speed of your internet connection and other factors (among them your UNIX prowess, but fear not). We recommend to download and install the packages to whatever location seem appropriate to you (if you sort your software into categories, for example), and then put symbolic links into the directory that contains this README, which we will refer to as PROOT (parrot root). This has the advantage of allowing us to abstract over version numbers and have generic names in PROOT. 1. OAA, the open agent architecture 1.1 Download OAA from http://www.ai.sri.com/~oaa . At the time of writing, version 2.3.1 is current; but hopefully later versions should work as well. Put a link to the directory into PROOT, and call it "oaa". 1.2 Copy setup1.pl from oaa/doc to your home directory, changing the name to setup.pl . At this point, you may want to change the line: default_facilitator(tcp(localhost, 3378)). to contain your actualy machine name, and if you prefer, a differnet port as well, e.g. default_facilitator(tcp('David-Schlangens-Computer.local', 55759)). 1.3 To test that the installation is working, run oaa/runtime/scripts/fac.sh This should give you messages that indicate whether the OAA facilitator could set up the agent network. If this didn't work: have fun with the OAA documentation... 1.4 Test the local setup: leave the facilitator running, and go to PROOT/StartItMod , which contains a slightly modified version of the config files for the OAA launcher GUI. To adapt the script to your setup, do the following: sed 's|PROOT|your_path_to_proot_here|' startit.sh_template \ > startit.sh This creates a localised version of startit.sh. Do the same with demo_unix.config: sed 's|PROOT|your_path_to_proot_here|' demo_unix.props_template \ > demo_unix.props You'll need the first part of this command a couple of times below, so you might want to put it somewhere where you can easily copy and paste it from. Now start startit.sh: ./startit.sh & This should bring up the launcher. Click on "OAA Monitor", "Start", this should bring up an agent that visualises your agent community, consisting so far of the faciliator, startit, and the monitor agent. ****"Debug Interface"->"Start" brings up a debugging console, which we will later use to test the setup.**** You can now exit ("kill"->"kill all and exit"). (For OAA experts: starting the facilitator manually seems to be necessary to get around certain bugs -- this may well have been fixed by the time you read this, so you could also use startit to launch the facilitator. But it doesn't hurt not to.) 2. sphinx4, the CMU Java speech recogniser 2.1 Download sphinx from http://cmusphinx.sourceforge.net/sphinx4/ and follow the instructions there for installation. Be sure to build all tests, and especially to setup the JSAPI, which comes with a special license and hence a special installer. (See sphinx documentation.) Put a link named "sphinx4" to the installation directory into PROOT. 2.2 Test it by running java -jar sphinx4/bin/HelloDigits.jar 3. trindikit, the ISU-based dialogue management kit 3.1 Download trindikit from: http://sourceforge.net/projects/trindikit/ At the time of writing, version 3.1.1 is the latest stable version; trindikit4 seems to have a different directory structure, so if you want to use that (once it becomes stable), you'll have to poke around a bit to find the file we require... ... file? well, actually, it's a bit odd, we will use dipper (a re-interpretation of the ISU theme) as our dialogue manager, but we need the trindikit oaa library from this distribution for the sphinx wrapper that will be installed in the next step. Anyway, download it, and put a symbolic link named "trindikit3" into PROOT. 4. the sphinx OAA wrapper 4.1 Download the wrapper from http://www.ling.gu.se/projekt/talk/software/wrappers.html (under "Sphinx 4 ASR"). Local name under PROOT: sphinx4agent 4.2 Configure the parrot-config for the agent: In PROOT/SphAgentMod, put in the correct paths. sed 's|PROOT|your_path_to_proot_here|' start.sh_template \ > start.sh Do the same for PROOT/SphAgentMod/config/digits/digits.config.xml_template . 4.3 OK, the first big moment. Start the facilitator (see 1.3 above) and the StartIt launcher (1.4), if they're not already running, and open the OAA Monitor. Now go to PROOT/SphAgentMod, and run start.sh -c config/digits/digits.config.xml [ By the way, you can very easily run the components on different machines -- that's the beauty of OAA. As long as the setup.pl on the client machine points to the machine and port where the facilitator is running, everything runs completely transparently. E.g., if you have copies of PROOT on all machines, and have adapted the setup.pl files on all machines, you can start the agents on any machine, and everything will still work exactly as described here. ] While this starts up, you should see the facilitator logging a contact attempt, and the monitor should show a "sphinx agent" icon. 4.4 Let's test this. Open the "Debug Interface" from StartIt, and type oaa_Solve(recognize(ReturnedResult), []) into the ICL Request field. The terminal where you started the SphinxAgent should show "Start speaking!", and after you've said something (only digits will be recognized, though), you should see a message in the Debug Interface window. Congratulations, you've just built your first (slightly) meaningful agent community! 5. freeTTS, a free Text-To-Speech synthesizer written in Java 5.1 Download freeTTS from http://freetts.sourceforge.net/ , install, test, and put a link to the installation directory in PROOT, name "freeTTS". 6. freeTTSagent, an OAA wrapper for freeTTS 6.1 download freeTTSagent from the Gothenburg trindikit page (see step 4.1 above) 6.2 adapth the paths in PROOT/freeTTSagMod: sed 's|PROOT|your_path_to_proot_here|' start_tts_agent.sh_template \ > start_tts_agent.sh 6.3 start the agent: freeTTSagMod/start_tts_agent.sh 6.4 open the Debug Interface again, and give it the following solvable: oaa_Solve(speakText('this is a test'), []) 7. dipper, the java-implementation of an ISU DM approach 7.1 download dipper_java, the java implementation of the dipper re-formulation of the trindikit-initiated ISU-based DM approach (still with us?), from http://www.inf.ed.ac.uk/research/isdd/admin/package?view=1&id=24 (If this page disappears, search for "Tetsushi Oka" (the author of this implementation) and "Dipper".) PROOT-local name for link: dipper_java 7.2 adapt paths in DipperMod (see above 6.2 etc.) 7.3 run PROOT/DipperMod/start_dipper_java.sh 7.4 load PROOT/DipperMod/Rules/parrot.is into dipper, as information state, and parrot.urules as update rules 7.5 make sure you have all the agents installed above running 7.5 hit "go!" in Dipper and .... there it is, your own personal, digits-repeating parrot!!! 8. look at parrot.is and parrot.urules, at the trindikit deliverables etc., and write your own dialogue managers that do more interesting things! (c) David Schlangen < das@ling.uni-potsdam.de > Use at your own risk. Have fun. ----- Versions: 15/07/06 - intial version ------ To Do: - set environment variable PROOT_HOME, use this in sed commands.... .... actually, could put all sed commands into script that is executed after everything has been downloaded.... this could be useful for adapting a full distribution that came on CD (for courses)... - add OAA wrapper for stanford parser... ----- local stuff: ln -s ../OAA/oaa2.3.1 oaa ln -s ../Speech/sphinx4-1.0beta-src/sphinx4-1.0beta sphinx4 ln -s ../DialMan/ISU_stuff/trindikit-3.2.2 trindikit3 ln -s ../OAA/Agents/sphinx4Agent sphinx4agent ln -s ../Speech/freetts-1.2.1 freeTTS ln -s ../OAA/Agents/freeTTSAgent freeTTSagent ln -s ../DialMan/ISU_stuff/dipper_java dipper_java sed 's|PROOT|/usr/local/src/NLP_stuff/proot|' start.sh_template > start.sh ------ making a distribution CD: - root dir: parrot_distr - copy all packages linked to into this dir; - copy proot into this dir; this ensures that symbolic links in proot keep working (they use relative paths) - copy everything from ~/DEAWU/..../Parrot into proot