pc28.am:Unity利用Sapi进行windows语音开垦,调节台语
分类:计算机编程

回想上高级中学时,给人超过生,支持买单月度薪资;用的正是带语音成效的总计器! 那个时候用起来倍儿爽,于是速度加倍,效用加快;结果让业主赔了累累钱!

软件中的语音技能首要满含两种:语音识别speech recognition和话音合成speech synthesis。日常地,开采者会因为技艺实力和资本实力等各个地方面的主题材料无力完毕标准的口音引擎,由此普通采纳现存的比较标准的语音引擎来达成有关的开采,举个例子国内特别著名的中国科学技术大学讯飞,百度语音等等。当然海外的还会有谷歌(Google卡塔尔语音,微软有SAPI等等。

      先天在天猫商城上订购了Kinect,刚刚到货,对于这一个新鲜的玩意儿,自个儿赶紧卸开包裹,插上PC机,先前早就装好了Kinect SDK(官方下载地址:)

正是因为那么些,才对语音总括器有了深入印象!恐怕是那货坑了自己!哼~!

在VTucson开采进程中,由于运转在Windows景况下,那么听天由命,大家首要推荐SAPI来扩充语音开辟。一是和Windows原生,二是离线没有供给网络,三是无需任何插件。别的就是SAPI发音,越发是保加利亚语发音,照旧相对来讲品质不错的。(Win7之上自带)

pc28.am 1

好啊,闲言少叙,直入正题吧!

使用SAPI,须求动用到的是System.Speech.dll文件。由于Unity须求将Dll文件放在Asset目录下,而如此的结果会发觉sapi failed to initialize。原因疑心为索要一定的上下文碰到才具运转dll的api,以致于拷贝到Asset目录引致上下文景况缺点和失误而不能够运行。

希望本身的电话能够跑得动DEMO,最终送了一口气,不荒谬运维:)。当然了,既然Kinect已经伊始,本身思忖也一丝一毫地练一入手,因为自身日前着重从事Web方面包车型大巴开拓,自然想到,如若能够在网页上可见运维Kinect该有多好哎!后来伪造了两种方案:

这几天在做二个类别,有个大约的成效,便是将文件转变来语音。

唯独要是做过那上边支出的敞亮,在C#的别的使用里面援引System.Speech.dll是全然没反常的。那么是或不是我们得以支付贰个专门的第三方程序,然后unity实行调用呢?依照这些思路,我们付出了二个调整台程序Speech.exe,主要功效是依靠输入文本举办语音合成。

 

切磋了那些作用后,抽空顺带做了个语音计算器!

代码较为轻巧

1)在Web应用程序上引用Kinect SDK的DLL(Microsoft.Research.Kinect.dll),能够正常援用,通过img标签恐怕页面输出图像流的办法展现Bitmap图像,当然你还亟需准期刷新页面,当然这种措施的宿疾简单来讲,正是相当不够实时性。而且扩张了服务器的承当。

“来银啊,上代码!”

/*简易的SAPI语音合成调节台程序*/

2)通过Silverlight应用程序的办法,可是近些日子SL4/5均不扶植Kinect的dll,这是出于Kinect是个.Net Framework的类库,非SL协理的客户端类库。其它,在非OOB的格局下,也SL近年来不扶持自定义的COM组件,希望微软共青团和少先队今后能够让SL也补助Kinect。

“老大,木有银,上不了”

using System.Speech.Synthesis;
using SpeechTest.Properties;

namespace SpeechTest
{
    class Program
    {
        static void Main(string[] args)
        {
            var speaker = new SpeechSynthesizer();
            speaker.Speak(“test”);
        }
    }
}

3)通过ActiveX的COM组件技巧,通过付出窗体组件,产生ActiveX插件,嵌套在IE浏览器中进行展示。

“哎哎我去,非逼小编说粗话,来个货,把代码码上来!”

OK,运营就足以听到机器发音Test了。

 

“好的,老大!”

大家改善一下,改为从参数中读取,那样的话,大家得以在unity中利用Process运维Speech.exe,并传给Speech参数。

      于是,小编思虑了最简便易行的方式,通过ActiveX的手艺在网页上对Kinect相关基本效能进行体现。

嘿嘿,笔者就是极其带点儿逗比的搬砖员儿!好呢,代码来了,小主,等急了呢?

/*从参数读取须要发声的文件*/

本文少禽分为多少个部分来陈诉:

using System;
using System.Speech.Synthesis;
using System.Collections.Generic;

namespace ReadTxt
{
    class Program
    {

        static string num1 = "", firchar = "", lastchar = "", tempRe = "";

        static void Main(string[] args)
        {

            Console.Title = "语音计算器";
            bool jump = true;
            do
            {

                ConsoleKeyInfo info = Console.ReadKey();
                switch (info.Key)
                {
                    case ConsoleKey.Escape: Environment.Exit(0); jump = false; break;
                    case ConsoleKey.NumPad0:    GetKeyRead("0");     break;
                    case ConsoleKey.NumPad1:    GetKeyRead("1");     break;
                    case ConsoleKey.NumPad2:    GetKeyRead("2");     break;
                    case ConsoleKey.NumPad3:    GetKeyRead("3");     break;
                    case ConsoleKey.NumPad4:    GetKeyRead("4");     break;
                    case ConsoleKey.NumPad5:    GetKeyRead("5");     break;
                    case ConsoleKey.NumPad6:    GetKeyRead("6");     break;
                    case ConsoleKey.NumPad7:    GetKeyRead("7");     break;
                    case ConsoleKey.NumPad8:    GetKeyRead("8");     break;
                    case ConsoleKey.NumPad9:    GetKeyRead("9");     break;
                    case ConsoleKey.Add:        GetKeyRead("加");    break;
                    case ConsoleKey.Subtract:   GetKeyRead("减");    break;
                    case ConsoleKey.Multiply:   GetKeyRead("乘");    break;
                    case ConsoleKey.Divide:     GetKeyRead("除");    break;
                    case ConsoleKey.Enter:
                        if (!string.IsNullOrEmpty(num1) && GetSignIsTrue(num1))
                        {
                            SetValue(num1);
                            num1 = "";
                        }
                        else
                        {
                            num1 = "";
                            if (!string.IsNullOrEmpty(num1))
                            {
                                Console.Beep();
                                Console.WriteLine("Error.");
                            }
                        }
                        break;
                    default:
                        break;
                }
            } while (jump);
            Console.Read();
        }

        //判断用户输入的内容是否合法
        static void GetKeyRead(string str)
        {
            SpeechSynthesizer spvoice = new SpeechSynthesizer();
            spvoice.Rate = 1;
            spvoice.Volume = 100;

            if (!string.IsNullOrEmpty(num1))
            {
                firchar = num1.Substring(0, 1);
                lastchar = num1.Substring(num1.Length - 1, 1);
            }

            switch (str)
            {
                case "加":
                    if (firchar != "加" && lastchar != "加")
                    {
                        if (lastchar != "减" && lastchar != "乘" && lastchar != "除")
                        {
                            num1  = str;
                        }
                        else
                        {
                            num1 = num1.Remove(num1.Length - 1);
                            num1  = str;
                        }
                    }

                    break;
                case "减":
                    if (firchar != "减" && lastchar != "减")
                    {
                        if (lastchar != "加" && lastchar != "乘" && lastchar != "除")
                        {
                            num1  = str;
                        }
                        else 
                        {
                            num1 = num1.Remove(num1.Length - 1);
                            num1  = str;
                        }
                    }

                    break;
                case "乘":
                    if (firchar != "乘" && lastchar != "乘")
                    {
                        if (lastchar != "加" && lastchar != "减" && lastchar != "除")
                        {
                            num1  = str;
                        }
                        else
                        {
                            num1 = num1.Remove(num1.Length - 1);
                            num1  = str;
                        }
                    }
                    break;
                case "除":
                    if (firchar != "除" && lastchar != "除")
                    {
                        if (lastchar != "加" && lastchar != "减" && lastchar != "乘")
                        {
                            num1  = str;
                        }
                        else
                        {
                            num1 = num1.Remove(num1.Length - 1);
                            num1  = str;
                        }
                    }
                    break;
                default:
                    num1  = str;
                    break;
            }
            spvoice.SpeakAsync(str);
            Console.Clear();
            Console.Write(tempRe   num1.Replace('加', ' ').Replace('减', '-').Replace('乘', '*').Replace('除', '/').Replace("等于", "="));
        }

        static bool GetSignIsTrue(string num1) 
        {
            if (!string.IsNullOrEmpty(num1))
            {
                firchar = num1.Substring(0, 1);
                lastchar = num1.Substring(num1.Length - 1, 1);
            }

            return firchar != "加" && lastchar != "加" && firchar != "减" && lastchar != "减" && firchar != "乘" && lastchar != "乘" && firchar != "除" && lastchar != "除";
        }

        //计算
        static void SetValue(string num1)
        {
            List<double> array = new List<double>();
            List<string> sign = new List<string>();
            num1 = num1.Replace('加', ' ').Replace('减', '-').Replace('乘', '*').Replace('除', '/');
            string tempCap = "";
            char[] MyChar = num1.ToCharArray();
            for (int i = 0; i < MyChar.Length; i  )
            {
                if (MyChar[i].ToString() == " " || MyChar[i].ToString() == "-" || MyChar[i].ToString() == "*" || MyChar[i].ToString() == "/")
                {
                    array.Add(Convert.ToDouble(tempCap));
                    tempCap = "";
                    sign.Add(MyChar[i].ToString());
                }
                else
                {
                    if (i == MyChar.Length - 1)
                    {
                        tempCap  = MyChar[i].ToString();
                        array.Add(Convert.ToDouble(tempCap));
                    }
                    else
                    {
                        tempCap  = MyChar[i].ToString();
                    }
                }
            }

            double resultTemp = 0;
            for (int i = 0; i < sign.ToArray().Length; i  )
            {
                if (sign[i] == "*")
                {
                    resultTemp = array[i] * array[i   1];
                    array[i] = resultTemp;
                    array.Remove(array[i   1]);
                    sign.Remove(sign[i]);
                    resultTemp = 0;
                    i--;
                }
                else if (sign[i] == "/")
                {
                    resultTemp = array[i] / array[i   1];
                    array[i] = resultTemp;
                    array.Remove(array[i   1]);
                    sign.Remove(sign[i]);
                    resultTemp = 0;
                    i--;
                }
            }

            for (int i = 0; i < sign.ToArray().Length; i  )
            {
                if (sign[i] == " ")
                {
                    resultTemp = array[i]   array[i   1];
                    array[i] = resultTemp;
                    array.Remove(array[i   1]);
                    sign.Remove(sign[i]);
                    resultTemp = 0;
                    i--;
                }
                else if (sign[i] == "-")
                {
                    resultTemp = array[i] - array[i   1];
                    array[i] = resultTemp;
                    array.Remove(array[i   1]);
                    sign.Remove(sign[i]);
                    resultTemp = 0;
                    i--;
                }
            }

            double answ = array[0];
            array.Clear();
            sign.Clear();
            tempRe  = num1.Replace('加', ' ').Replace('减', '-').Replace('乘', '*').Replace('除', '/')   "= "   answ   "n";
            GetKeyRead("等于"   answ);
        }
    }
}
using System.Speech.Synthesis;
using SpeechTest.Properties;

namespace SpeechTest
{
    class Program
    {
        static void Main(string[] args)
        {
            var speaker = new SpeechSynthesizer();
            var res = args.Length == 0 ? "请说" : args[0];
            speaker.Speak(res);
        }
    }
}

豆蔻梢头、ActiveX插件的贯彻;

ok,那正是全方位代码了,请容在下粗略的介绍下达成步骤:

大家先利用CMD命令行,cd到Speech.exe所在的目录,然后输入Speech.exe test,如笔者辈预料的那么,机器发音test。测验通过。

二、Kinect根底效的落实;

1.既然是语音计算器,首先要消除的便是哪些落到实处语音?

为了能够转移发音的布置,增添部分代码,从Setting中读取相关的配置数据,代码改进如下:

三、ActiveX的安装;

事实上,语音效率完结起来并不复杂,微软早已封装好了三个,将文件转变到语音功能的类库,未来你要做的就是:找到它,然后把它引用到你的项目里!

/*能够配置的调节台程序*/

 

怎样?那几个类库叫什么?

using System.Speech.Synthesis;
using SpeechTest.Properties;

namespace SpeechTest
{
    class Program
    {
        static void Main(string[] args)
        {
            var speaker = new SpeechSynthesizer();
            speaker.Volume = Settings.Default.SpeakVolume;
            speaker.Rate = Settings.Default.SpeakRate;
            var voice = Settings.Default.SpeakVoice;
            if (!string.IsNullOrEmpty(voice))
                speaker.SelectVoice(voice);
            var res = args.Length == 0 ? "请说" : args[0];
            speaker.Speak(res);
        }
    }
}

现实完成

表急嘛,心急吃不了“热水豆腐”,有异常的大恐怕“水豆腐”还跑了呢,要耐性点儿,技艺约到手!是啊?

接下去大家在Unity中央银行使Process来张开那个Speech.exe,代码如下:

首先局地  ActiveX插件的兑现

好吗,介绍下这几个类库,它叫“System.Speech.dll”。.net FrameWork 的设置目录里就有它的人影!装了.net FrameWork 的童鞋,能够因而以下目录去找寻它!

/*Unity中开启Speech.exe进程*/

1) 成立二个新的消除方案,叫做MyFirstKinect。

大假如其风度翩翩:C:Program Files (x86)Reference AssembliesMicrosoftFramework.NETFrameworkv4.5

using System.Diagnostics;

public class Speecher: MonoBehaviour
{
    public static void Speak(string str)
    {
        var proc = new Process
        {
            StartInfo = new ProcessStartInfo
            {
                FileName = "speech.exe",
                Arguments = """   str   """,
            }
        };
        proc.Start();
    }

    /***测试代码,可删除Start***/
    protected void Start()
    {
        Speak("test");
    }
    /***测试代码,可删除End***/
}

 

寻寻找觅,销声敛迹,凄悲戚惨戚戚!什么?没寻找到?

将脚本挂在别的二个GO(GameObject)上,运维,黑框现身,同时听到发音,测验完了。

2)接着创制二个Windows窗体控件库,用于做ActiveX的插件,项目名叫MyFirstKinectControl

好呢,教您生龙活虎招

接下去大家隐瞒那么些黑框。代码修正如下:

pc28.am 2

pc28.am 3

/*Unity开启无框的Speech.exe进度*/

 

 

using System.Diagnostics;

public class Speecher: MonoBehaviour
{
    public static void Speak(string str)
    {
        var proc = new Process
        {
            StartInfo = new ProcessStartInfo
            {
                FileName = "speech.exe",
                Arguments = """   str   """,
                CreateNoWindow = true,
                WindowStyle = ProcessWindowStyle.Hidden,
            }
        };
        proc.Start();
    }
    /***测试代码,可删除Start***/
    protected void Start()
    {
        Speak("test");
    }
    /***测试代码,可删除End***/
}

3)在MyFirstKinectControl项目标右键点击“属性”,采取“生成”项:

开辟C盘,直接搜索!

实质上到了这一步,首要的功用都形成了。可是稳重的会开掘,那样持续制程然后停业进度的办法会不会太笨了。可不得以让Speech那么些进度平昔张开着,收到unity的音讯时就嚷嚷呢?那就事关到进度间通讯了。

pc28.am 4

没装的,那就去网络搜吧,自个儿下崽儿!哈哈~!愿意装的也得以装一下!

Windows的进程是互相独立的,各自有分别的分红空间。可是并不表示这不能够互相通讯。方法有非常多,举个例子读写文件,发送新闻(hook),Socket等等。个中Socket完毕起来相对轻松,极度是我们曾经持有Socket封装库的情事下,只要一丢丢代码就能够了。

将”为COM互操作注册”勾上,然后关门。

找到的,直接引用到品种里就能够了哈!

于是在Speech改成二个Socket服务器,代码如下:

 

2.其次,别忘了征引下那八个命名空间:

/*Speech 服务端*/

4)打开AssemblyInfo.cs:

  using System.Speech.Synthesis;  //语音类库

using System;
using System.Linq;
using System.Speech.Synthesis;
using System.Text;
using Speech.Properties;

namespace Speech
{
    class Program
    {
        static void Main(string[] args)
        {
            var server = new NetServer();
            server.StartServer();

            while (true)
            {
                var res = Console.ReadLine();
                if (res == "exit")
                    break;
            }
        }
    }

    public class NetServer : SocketExtra.INetComponent
    {
        private readonly Speecher m_speecher;

        private readonly SocketExtra m_socket;

        public NetServer()
        {
            m_speecher = new Speecher();
            m_socket = new SocketExtra(this);
        }

        public void StartServer()
        {
            m_socket.Bind("127.0.0.1", Settings.Default.Port);
        }

        public bool NetSendMsg(byte[] sendbuffer)
        {
            return true;
        }

        public bool NetReciveMsg(byte[] recivebuffer)
        {
            var str = Encoding.Default.GetString(recivebuffer);
            Console.WriteLine(str);
            m_speecher.Speak(str);
            return true;
        }

        public bool Connected { get { return m_socket.Connected; } }
    }

    public class Speecher
    {
        private readonly SpeechSynthesizer m_speaker;

        public Speecher()
        {
            m_speaker = new SpeechSynthesizer();
            var installs = m_speaker.GetInstalledVoices();

            m_speaker.Volume = Settings.Default.SpeakVolume;
            m_speaker.Rate = Settings.Default.SpeakRate;
            var voice = Settings.Default.SpeakVoice;

            var selected = false;
            if (!string.IsNullOrEmpty(voice))
            {
                if (installs.Any(install => install.VoiceInfo.Name == voice))
                {
                    m_speaker.SelectVoice(voice);
                    selected = true;
                }
            }
            if (!selected)
            {
                foreach (var install in installs.Where(install => install.VoiceInfo.Culture.Name == "en-US"))
                {
                    m_speaker.SelectVoice(install.VoiceInfo.Name);
                    break;
                }
            }
        }

        public void Speak(string msg)
        {
            m_speaker.Speak(msg);
        }
    }
}

pc28.am 5

  using System.Collections.Generic;  //数组群集类库

相同的时候改革Unity代码,扩展Socket相关代码:

将ComVisible设置为true,并将上边那行注释掉(那么些超级重大,切记!)

3.结尾,简介下文件调换来语音的功效!

/*Unity客商端代码*/

对应地,必要在自定义窗体控件上增加该Guid:

实在,也就几句话!

using System.Collections;
using System.Diagnostics;
using System.Text;
using UnityEngine;

public class Speecher : MonoBehaviour, SocketExtra.INetComponent
{
    private SocketExtra m_socket;
    private Process m_process;

    protected void Awake()
    {
        Ins = this;
        m_process = new Process
        {
            StartInfo = new ProcessStartInfo
            {
                FileName = "speech.exe",
                CreateNoWindow = true,
                WindowStyle = ProcessWindowStyle.Hidden
            },
        };
        m_process.Start();
    }

    /***测试代码,可删除Start***/
    protected IEnumerator Start()
    {
        yield return StartCoroutine(Connect());
        Speak("test");
    }
    /***测试代码,可删除End***/

    public IEnumerator Connect()
    {
        m_socket = new SocketExtra(this);
        m_socket.Connect("127.0.0.1", 9903);
        while (!m_socket.Connected)
        {
            yield return 1;
        }
    }

    protected void OnDestroy()
    {
        if (m_process != null && !m_process.HasExited)
            m_process.Kill();
        m_process = null;
    }

    public static Speecher Ins;

    public static void Speak(string str)
    {
#if UNITY_EDITOR||UNITY_STANDALONE_WIN
        Ins.Speech(str);
#endif
    }

    public void Speech(string str)
    {
        if (m_socket.Connected)
        {
            var bytes = Encoding.Default.GetBytes(str);
            m_socket.SendMsg(bytes);
        }
    }

    public bool NetReciveMsg(byte[] recivebuffer)
    {
        return true;
    }

    public bool NetSendMsg(byte[] sendbuffer)
    {
        return true;
    }
}

pc28.am 6

SpeechSynthesizer spvoice = new SpeechSynthesizer();  //语音类
spvoice.Rate = 1;            //使用 spvoice 设置朗读频率 [范围 -10 ~ 10] 
spvoice.Volume = 80;         //使用 spvoice 设置朗读音量 [范围 0 ~ 100] 
spvoice.SpeakAsync("大家好,我是博主小白!");   //开始读

OK,大功告成。工程见Githubpc28.am 7

由来,叁个为主的COM组件已经落到实处了,未来来看下该怎么在浏览器上展现ActiveX插件。

好了,其余的代码将在看小主的内力了,在下就相当的少说了!

 

看小主骨骼惊喜,“英语优越”,想是定能习得搬砖精粹,打遍天下须要,杀掉红尘bug;然后走上淫僧顶峰,赢取大白靓女的!哈哈哈~!

转发请注脚出处www.codegize.com

5)张开Visual Studio的命令提醒符:输入“oleview”,页面会展开三个“OLE/COM Object Viewer”应用程序:

 

pc28.am 8

拙文意气风发篇,望各位海涵!

是因为作者是使用C#开创的COM组件,于是在“.NET Category”寻觅刚才成立的”MyFirstKinectControl”:

技术研讨群:225443677 有意者迎接打扰,多谢!

pc28.am 9

 

右键选用“Copy HTML<object> Tag to Clipboard”,得到:

pc28.am 10

 

6)然后自身在自定义窗体控件上(SkeletalControl.cs),随意步入点东西上去,举例开关、标签等等。

 

7)然后新成立三个Web应用程序的品种(WebApp),重新编写翻译。将上边的代码复制到Html或相关页面中。

pc28.am 11

在IE符合规律状态下,开掘插件不能日常显示。于是,把浏览器的安全等第调节减弱:

pc28.am 12

世襲运转:

pc28.am 13

就能够经常展现插件了。当然这种方式形成了浏览器选拔上的危机性,所以不提出如此来利用。

 

8)假如想要在不调解浏览器安全级其余场地下,又能够在浏览器上符合规律呈现插件,那样就非得调解一些代码:

[Guid("CB5BDC81-93C1-11CF-8F20-00805F2CD064"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
public interface IObjectSafety
{
    // methods 
    void GetInterfacceSafyOptions(
        System.Int32 riid,
        out System.Int32 pdwSupportedOptions,
        out System.Int32 pdwEnabledOptions);
    void SetInterfaceSafetyOptions(
        System.Int32 riid,
        System.Int32 dwOptionsSetMask,
        System.Int32 dwEnabledOptions);
}

增进多少个IObjectSafety的接口,并且Guid是稳固的。

SkeletalControl继承这些接口:

[Guid("d678c286-b26f-4f72-ae22-2dcb1952851b")]
public partial class SkeletalControl : UserControl, IObjectSafety
{
    public SkeletalControl()
    {
        InitializeComponent();
    }

    #region IObjectSafety 成员

    public void GetInterfacceSafyOptions(Int32 riid, out Int32 pdwSupportedOptions, out Int32 pdwEnabledOptions)
    {
        pdwSupportedOptions = 1;
        pdwEnabledOptions = 2;
    }

    public void SetInterfaceSafetyOptions(Int32 riid, Int32 dwOptionsSetMask, Int32 dwEnabledOptions)
    {

    }

    #endregion
}

紧接器重新编写翻译并运维Web程序,运营结果为:

pc28.am 14

诸如此比,你就无需调动浏览器的安全品级,就足以健康展现ActiveX插件了。

 

其次片段 Kinect的功底用实现

 

      从这一有的起,作者将上马介绍Kinect怎样落实部分根底用:满含录制监察和控制、骨骼追踪以至声音控制截屏的法力。

1)项目中引用以下的Dll:

pc28.am 15

内部Microsoft.Research.Kinect正是在Computer上装好Kinect SDK后方可援用的类库;

此内地,Coding4Fun.Kinect.WinForm是二个依据SDK的DLL的相干封装好的有些效果与利益类库,网络开源地址为:;

Microsoft.Speech是二个微软提供的话音识别的中坚类库,也蕴涵相关的SDK,何况和Kinect进行绑定的连带类库,具体地址在Kinect SDK中的相关文书档案也可以有证实:

- Speech Platform Runtime (v10.2) x86. Even on x64 platforms the x86 needs to be used because the MSR Kinect SDK runtime is x86
 

  • Speech Platform SDK (v10.2)
     
  • Kinect English Language Pack: MSKinectLangPack_enUS.msi (available in the same location as the Kinect For Windows SDK)

 

2)在控件页面上创设三个PictureBox的控件:

pc28.am 16

多少个图片框将分别用来存放:深度图录制、普通录像、以致骨骼追踪。

 

3)编写相关代码:

using Microsoft.Research.Kinect.Nui;
using Coding4Fun.Kinect.WinForm;

Runtime nui;

private void SkeletalControl_Load(object sender, EventArgs e)
{
    nui = new Runtime();

    try
    {
        nui.Initialize(
            RuntimeOptions.UseDepthAndPlayerIndex 
            | RuntimeOptions.UseSkeletalTracking 
            | RuntimeOptions.UseColor);
    }
    catch (InvalidOperationException)
    {
        MessageBox.Show("Runtime initialization failed. Please make sure Kinect device is plugged in.");
        return;
    }

    try
    {
        nui.VideoStream.Open(ImageStreamType.Video, 2, ImageResolution.Resolution640x480, ImageType.Color);
        nui.DepthStream.Open(ImageStreamType.Depth, 2, ImageResolution.Resolution320x240, ImageType.DepthAndPlayerIndex);
    }
    catch (InvalidOperationException)
    {
        MessageBox.Show("Failed to open stream. Please make sure to specify a supported image type and resolution.");
        return;
    }

    nui.DepthFrameReady  = new EventHandler(nui_DepthFrameReady);
    nui.SkeletonFrameReady  = new EventHandler(nui_SkeletonFrameReady);
    nui.VideoFrameReady  = new EventHandler(nui_VideoFrameReady);
}

里头DepthFrameReady,VideoFrameReady,SkeletonFrameReady分别用来追踪深度图、普通视图、骨骼图所发生的事件。

void nui_DepthFrameReady(object sender, ImageFrameReadyEventArgs e)
 {
     pictureBoxDepth.Image = e.ImageFrame.ToBitmap();
 }

 void nui_VideoFrameReady(object sender, ImageFrameReadyEventArgs e)
 {
     pictureBoxVideo.Image = e.ImageFrame.ToBitmap();
 }

 void nui_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
 {
     Graphics graphics = this.pictureBoxSkeleton.CreateGraphics();

     pictureBoxSkeleton.Refresh();

     SkeletonFrame skeletonFrame = e.SkeletonFrame;
     int iSkeleton = 0;

     foreach (SkeletonData data in skeletonFrame.Skeletons)
     {
         if (SkeletonTrackingState.Tracked == data.TrackingState)
         {
             // Draw bones
             graphics.DrawLines(pen, getBodySegment(data.Joints, JointID.HipCenter, JointID.Spine, JointID.ShoulderCenter, JointID.Head));
             graphics.DrawLines(pen, getBodySegment(data.Joints, JointID.ShoulderCenter, JointID.ShoulderLeft, JointID.ElbowLeft, JointID.WristLeft, JointID.HandLeft));
             graphics.DrawLines(pen, getBodySegment(data.Joints, JointID.ShoulderCenter, JointID.ShoulderRight, JointID.ElbowRight, JointID.WristRight, JointID.HandRight));
             graphics.DrawLines(pen, getBodySegment(data.Joints, JointID.HipCenter, JointID.HipLeft, JointID.KneeLeft, JointID.AnkleLeft, JointID.FootLeft));
             graphics.DrawLines(pen, getBodySegment(data.Joints, JointID.HipCenter, JointID.HipRight, JointID.KneeRight, JointID.AnkleRight, JointID.FootRight));
         }
         iSkeleton  ;
     } // for each skeleton
 }

 private Point getDisplayPosition(Joint joint)
 {
     float depthX, depthY;
     nui.SkeletonEngine.SkeletonToDepthImage(joint.Position, out depthX, out depthY);
     depthX = Math.Max(0, Math.Min(depthX * 320, 320));  //convert to 320, 240 space
     depthY = Math.Max(0, Math.Min(depthY * 240, 240));  //convert to 320, 240 space
     int colorX, colorY;
     ImageViewArea iv = new ImageViewArea();
     // only ImageResolution.Resolution640x480 is supported at this point
     nui.NuiCamera.GetColorPixelCoordinatesFromDepthPixel(ImageResolution.Resolution640x480, iv, (int)depthX, (int)depthY, (short)0, out colorX, out colorY);

     // map back to skeleton.Width & skeleton.Height
     return new Point((int)(this.pictureBoxSkeleton.Width * colorX / 640.0), (int)(this.pictureBoxSkeleton.Height * colorY / 480));
 }

 Point[] getBodySegment(Microsoft.Research.Kinect.Nui.JointsCollection joints, params JointID[] ids)
 {
     Point[] points = new Point[ids.Length];
     for (int i = 0; i < ids.Length;   i)
     {
         points[i] = getDisplayPosition(joints[ids[i]]);
     }

     return points;
 }

中间,getBodySegment,getDisplayPosition方法将显明骨骼追踪中的十几个骨骼点的具体地方。

 

4)接着编写翻译并运转程序,查看Web页面,连上Kinect传感设备,运维结果为:
pc28.am 17

 

5)接着,来实现部分声音控制截屏功能:

using Microsoft.Research.Kinect.Audio;
using Microsoft.Speech.AudioFormat;
using Microsoft.Speech.Recognition;



private const string RecognizerId = "SR_MS_en-US_Kinect_10.0";
private KinectAudioSource kinectSource;
private SpeechRecognitionEngine sre;



// 声控截屏功能
RecognizerInfo ri = SpeechRecognitionEngine.InstalledRecognizers().Where(r => r.Id == RecognizerId).FirstOrDefault();
if (ri == null)
{
    MessageBox.Show("Could not find speech recognizer: {0}. Please refer to the sample requirements.", RecognizerId);
    return;
}

sre = new SpeechRecognitionEngine(ri.Id);

var colors = new Choices();
colors.Add("cut"); //添加cut的英文发音

var gb = new GrammarBuilder();                          
gb.Culture = ri.Culture; //本地化处理
gb.Append(colors);

var g = new Grammar(gb);

sre.LoadGrammar(g);
sre.SpeechRecognized  = SreSpeechRecognized;                    //发音匹配以后的后续处理事件
sre.SpeechHypothesized  = SreSpeechHypothesized;                //发音的英文识别事件
sre.SpeechRecognitionRejected  = SreSpeechRecognitionRejected;  //拒绝之后的后续处理事件

var thread = new Thread(StartDMO);
thread.Start();




private void StartDMO()
 {
     kinectSource = new KinectAudioSource();
     kinectSource.SystemMode = SystemMode.OptibeamArrayOnly;
     kinectSource.FeatureMode = true;
     kinectSource.AutomaticGainControl = false;
     kinectSource.MicArrayMode = MicArrayMode.MicArrayAdaptiveBeam;
     var kinectStream = kinectSource.Start();
     sre.SetInputToAudioStream(kinectStream, new SpeechAudioFormatInfo(
                                           EncodingFormat.Pcm, 16000, 16, 1,
                                           32000, 2, null));
     sre.RecognizeAsync(RecognizeMode.Multiple);
 }

 void SreSpeechRecognitionRejected(object sender, SpeechRecognitionRejectedEventArgs e)
 {

 }

 void SreSpeechHypothesized(object sender, SpeechHypothesizedEventArgs e)
 {

 }

 void SreSpeechRecognized(object sender, SpeechRecognizedEventArgs e)
 {
     lblSpeech.Text = e.Result.Text;

     //屏幕截屏
     Bitmap bmp = new Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height);
     Graphics g = Graphics.FromImage(bmp);
     g.CopyFromScreen(0, 0, 0, 0, bmp.Size);
     g.Dispose();

     SaveFileDialog fileDialog = new SaveFileDialog();
     fileDialog.Filter = "JPG File(*.jpg)|*.jpg||";
     DialogResult result = fileDialog.ShowDialog();
     if (result == DialogResult.OK)
     {
         bmp.Save(fileDialog.FileName, System.Drawing.Imaging.ImageFormat.Jpeg);
     }
 }

在代码中窥见,var colors = new Choices(卡塔尔(英语:State of Qatar); colors.Add("cut"卡塔尔(英语:State of Qatar); //增加cut的法语发音 ,这样当您在传感器前失声“cut”就能展开连锁事件的触发,当发音和德文库的单词语音识别向相配时,将触发SreSpeechRecognized事件。

试行荧屏截图的连锁操作。

(注:记得这里亟待增加代码gb.Culture = ri.Culture; 若无那句代码,有望形成sre.LoadGrammar(g卡塔尔国; 语法加载退步!)

 

  1. 运维结果:

pc28.am 18

自己将刚才发的România语单词,通过文件的艺术显示在页面中。

 

其三片段 ActiveX插件的安装

      由于本文使用的是C#来开辟ActiveX插件,所以当您必要安装插件的时候,须求动用re瓦斯m命令。那么伊始工编织制脚本:

1)安装脚本:

@echo off
echo 开始安装MyFirstKinectControl......
echo.
set FrameworkPath=%SystemRoot%Microsoft.NETFrameworkv4.0.30319
if exist "%FrameworkPath%regasm.exe" goto :Start
set FrameworkPath=%SystemRoot%Microsoft.NETFrameworkv3.5
if exist "%FrameworkPath%regasm.exe" goto :Start
set FrameworkPath=%SystemRoot%Microsoft.NETFrameworkv3.0
if exist "%FrameworkPath%regasm.exe" goto :Start
set FrameworkPath=%SystemRoot%Microsoft.NETFrameworkv2.0.50727
if exist "%FrameworkPath%regasm.exe" goto :Start
set FrameworkPath=%SystemRoot%Microsoft.NETFrameworkv1.1.4322
if exist "%FrameworkPath%regasm.exe" goto :Startv
set FrameworkPath=%SystemRoot%Microsoft.NETFramework1.0.3705
if exist "%FrameworkPath%regasm.exe" goto :Start

:Start
%FrameworkPath%regasm.exe MyFirstKinectControl.dll /codebase MyFirstKinectControl.dll

echo 安装完成!
echo.

pause

 

2)卸载脚本:

@echo off
echo 开始卸载MyFirstKinectControl......
echo.
set FrameworkPath=%SystemRoot%Microsoft.NETFrameworkv4.0.30319
if exist "%FrameworkPath%regasm.exe" goto :Start
set FrameworkPath=%SystemRoot%Microsoft.NETFrameworkv3.5
if exist "%FrameworkPath%regasm.exe" goto :Start
set FrameworkPath=%SystemRoot%Microsoft.NETFrameworkv3.0
if exist "%FrameworkPath%regasm.exe" goto :Start
set FrameworkPath=%SystemRoot%Microsoft.NETFrameworkv2.0.50727
if exist "%FrameworkPath%regasm.exe" goto :Start
set FrameworkPath=%SystemRoot%Microsoft.NETFrameworkv1.1.4322
if exist "%FrameworkPath%regasm.exe" goto :Startv
set FrameworkPath=%SystemRoot%Microsoft.NETFramework1.0.3705
if exist "%FrameworkPath%regasm.exe" goto :Start

:Start
%FrameworkPath%regasm.exe /u MyFirstKinectControl.dll

echo 卸载完成!
echo.

pause

 

那样经过注册COM组件就能够兑现Kinect的插件在浏览器上的显得。

 

依靠本文的源代码:MyFirstKinect.rar   多谢我们阅读!

本文由pc28.am发布于计算机编程,转载请注明出处:pc28.am:Unity利用Sapi进行windows语音开垦,调节台语

上一篇:将Log4net的配置配置到的独立文件中,Log4Net日志插 下一篇:没有了
猜你喜欢
热门排行
精彩图文