计算机视觉专题

计算机视觉专题

TableofContents计算机视觉专题•IntroductionTopicsonComputerVision•VisualInference•BayesianTracking:KF&PF视觉运动分析•VisualFeatureExtractionVisualMotionAnalysis•Multi-targetTracking:DataAssociation•HumanPoseEstimationSummer,200712Outline第二讲•Introduction•Graphicalmodels视觉推理实例、理论与工具•StatespacemodelandBayesianrule•Challengesinvisualmotionanalysis(VMA)?2007-7-19•HowtodoresearchonVMA?34OutlineWhatisthegoalofvision?•Introduction看图说话•GraphicalmodelsSemanticLabel•StatespacemodelandBayesianruleOutput•ChallengesinvisualmotionanalysisInput?+(VMA)?•HowtodoresearchonVMA?NumericalInformation561\nStateSpaceModelWhatispossibletoday?Statespace(discretelabel;continuousnumericalvalue)•Someofeach,allofnoneObservationspace(imagefeaturespace)•SystemsworkunderlimitedconditionsObservationmappingfunctionF–Limitednumberofobjects–Controlledorstructuredenvironmentsy0y1y2–ControlledimagingconditionsΘ–Forlimitedconditions,performancemay−1cF:RcSΘF:RΘSbebetterthanforhumans,forgeneral−1cyx=F()FF()yx=∈{}R()xy=situations,performanceismuchweakercRthanforhumansx2x3x0xx4781Discriminativevs.GenerativeDiscriminativevs.GenerativeIfyouareasking,Ifyouareasking,“Arethereanyfacesinthis“Arethereanyfacesinthisimage?”,image?”,thenyouwouldprobablythenyouwouldprobablywanttousediscriminativewanttousediscriminativemethods.methods.OpenCV中文网站Ifyouareasking,“Finda3-dmodelthathttp://www.opencv.org.cndescribestherunner”,AdaBoost人脸检测(Slide)thenyouwouldusehttp://www.opencv.org.cn/images/3/34/generativemethods.OpenCV_FaceDetection_June10.pdf910实例：MotionAnalysis初始化行为分析轨迹/区域跟踪事件判别姿态重建触发执行器11122\nHumanMotionTrackingTrackingbasics•ObjectTracking•人体运动跟踪–Generatethetrajectoryofanobjectovertime–Trajectory/region–Biggestproblem--DataAssociation•全局运动•整体作为单个目标•Keyideas–Posture（姿态）–Trackingbydetection•局部运动–Trackingthroughflow/apperance•关节式物体各个子部分之间的连接关系，即关节角度•由层级式的骨骼结构来描述1314Trackbydetection•Assume–afairlyreliabledetector(e.g.faces;backofheads)–detectionsthatarewellspacedinimages(orhavedistinctiveproperties)•Trackby–detectineachframe–linkusingmatchingalgorithm•measurementswithnotrack?createnewtrack•trackswithnomeasurement?wait,thenreap1516Trackbyflow/appearanceMean-ShiftObjectTrackingGeneralFramework:TargetRepresentation•Assume–appearanceunknown(butdomain,parametricflowmodelknown)ChooseaRepresentthe•equivalently,deformationorcorrespondencemodelreferenceChooseamodelinthe•flowispixel-pixelcorrespondencemodelinthefeaturespacechosenfeature–Appearancemodelingcurrentframespace•Initialize–markoutdomain•Track–chooseflowmodelparametersthataligndomaininpicnwithn+1best–pushdomainthroughflowmodeltogetnextdomain•KLT,Mean-shift•http://www.opencv.org.cn/--运动分析与对象跟踪……Current•CalcOpticalFlowLK,MeanShift,CamShift17frame183\nMean-ShiftObjectTrackingMean-ShiftObjectTrackingGeneralFramework:TargetLocalizationTargetRepresentationStartfromtheSearchintheFindbestpositionofthemodel’scandidatebyChooseaRepresentthemodelintheneighborhoodmaximizingareferenceChooseamodelbyitscurrentframeinnextframesimilarityfunc.targetmodelfeaturespacePDFinthefeaturespace0.35Repeatthe0.3sameprocessQuantized0.25inthenextpair0.2ColorSpace0.15offramesProbability0.10.050123...mcolorModelCandidateCurrent……1920frameKernelBasedObjectTracking,byComaniniu,Ramesh,MeerMean-ShiftObjectTrackingRecallSSMPDFRepresentationStatespace(discretelabel;continuousnumericalvalue)TargetModelTargetCandidate(centeredat0)(centeredaty)Observationspace(imagefeaturespace)0.350.3ObservationmappingfunctionF0.30.250.250.20.20.150.15ProbabilityProbability0.1y0.10yy120.050.0500123...m123...mcolorcolorΘdmdm−1cF:RcSΘF:RΘSqq=={}uuum=1..∑q1py()=={}pyuu()um=1..∑p1u=1u=1yx=F()FF−1()yx=∈{}Rc()xy=cRSimilarityddx2xf()yfq=⎡⎤⎣⎦,py()3Function:21x0xx4221PoseEstimationandTrackingPoseEstimationandTrackingß手工标定关节点位置，逐点、逐线匹配方法ß基于模式分类的姿态识别方法[Wu2001,Hamdan1999,ß基于逆运动学链计算的估计方法Wu2000,Weng1997,Moghaddam1995]ß基于二维/三维运动学模型或几何模型的优化方法ß基于回归分析的三维姿态估计方法[Rosales2000，Rosales－手工抽取整体骨架[Guo1991，Luo1992]2001]－采用启发式规则，提取手腕、指尖、手指边缘中线－获取不同视角不同姿态下的训练样本，学习二维观测与三维或手指边缘[Hogg1983，Rehg1994，Wu1999]姿态之间的对应关系，将学习得到的决策规则或回归函数应用于样本，所得结果做为对样本的姿态估计ß基于样本库的最近邻查找问题[Shimada2001，Athitsos2003]缺点：需要大量的采样密集的训练数据，且无法提供精确的估计值缺点：估计精度依赖细节特征的抽取，或需要比较准确的预测值23244\nModel-basedTracking•Analysis-by-Synthesis2526RecallSSM预测值Statespace(discretelabel;continuousnumericalvalue)图像序列Observationspace(imagefeaturespace)估计值ObservationmappingfunctionFy0y1y特征匹配2当前图像预测仿真图像Θ−1cF:RcSΘF:RΘS−1cyx=F()FF()yx=∈{}R()xy=cRx图像特征预测图像特征2x327x0xx4281DifficultiesinmotionanalysisOutline•Lossofinformation3D->2D•Introduction•Noiseinimages•Graphicalmodels•Complexobjectmotion•StatespacemodelandBayesianrule•Nonrigidorarticulatednatureofobjects•Challengesinvisualmotionanalysis•Partialorfullobjectocclusion(VMA)?•Complexobjectshapes•HowtodoresearchonVMA?•Sceneilluminationchanges•Realtimeprocessingrequirements29305\nRecallSSMMarkov链Statespace(discretelabel;continuousnumericalvalue)•一般及常用的统计中，彼此相互「独立」大概是Observationspace(imagefeaturespace)最有用的一個观念。用简单的术语來说，互相「独立」就是彼此毫不相干，一點牵涉都沒有。ObservationmappingfunctionF•但是实际生活中很多事件是相互关联的y0y1y2•[不是互相獨立」也就是說互相关联的意思，但是要怎样相关呢？如何在相关中作一些简单的分类Θ呢？馬可夫连就是要描述在「相关」這个概念中−1cF:RcSΘF:RΘS最简单的一种。但即使如此，有关马可夫链的理−1cyx=F()FF()yx=∈{}R()xy=论已经相当丰富了。在概率理論中，它几乎占了cR绝大的部分。x2x3x0xx431321马可夫链Bayesiannetworks•在马可夫鏈中我們考虑最简单的「相关」性。在•Directedacyclicgraph(DAG)EarthquakeBurglary在这种情况下，我们不能给任一個事件E一個概–Nodes–randomvariablesj–Edges–directinfluence率pj但我们给一对事件(Ej,Ek)一個概率pjk，这个(“causation”)时候p的解释是一种条件概率，就是假设事件Ejkj•Xi?Xancestors|XparentsRadioAlarm已经出现，则E出現的概率。除了p之外，我們kjk•e.g.,C?{R,B,E}|A还需要知道E出現的機率a。有了这些资料后，jj•Simplifieschainrulebyusing一個样本序列Ej0Ej1…Ejn（也就是说第零个事件conditionalindependenciesCall是E，第一个是E……第n个是E）的概率就很j0j1jn清楚的是P(E,E,E)=app…p。j0j1jnjj0j1j1j2jn-1jn3334Pearl,1988隐马可夫模型定义•但是在大多数情况下我们所观察到的值并不一个隐马尔可夫模型(HMM)是一个五元组：是序列本身的元素。(ΩX,ΩO,A,B,π)•即观察值不等于状态值。其中：•故我们隐入马可夫模型。ΩX={q1,...qN}：状态的有限集合ΩO={v1,...,vM}：观察值的有限集合A={aij}，aij=p(Xt+1=qj|Xt=qi)：转移概率B={bik}，bik=p(Ot=vk|Xt=qi)：输出概率π={πi}，πi=p(X1=qi)：初始状态分布35366\n假设HiddenMarkovModel对于一个随机事件，有一个观察值序列：O1,...,OT•Examples该事件隐含着一个状态序列：X1,...,XT假设1：马尔可夫假设（状态构成一阶马尔可夫链）p(Xi|Xi-1…X1)=p(Xi|Xi-1)X1XXhidden假设2：不动性假设（状态与具体时间无关）23p(Xi+1|Xi)=p(Xj+1|Xj)，对任意i,j成立假设3：输出独立性假设（输出仅与当前状态有关）Y1Y2Y3observedp(O1,...,OT|X1,...,XT)=Πp(Ot|Xt)3738State-spaceModelMarkovRandomFields•Grids(Pixels,Patches,Objects)2Dstructure,undirectedgraph3940MarkovRandomFieldsMRFnodesaspatches•MarkovRandomFieldsbreak1DstructureofHMMimage–Fieldofsites,eachofwhichhasalabel,simultaneously.patches–Labelatonesitedependentonothers,no1Dstructuretodependencies.–Thismeansnooptimal,efficientalgorithms.sceneimagepatches•WhyMRFs?Φ(xi,yi)–Objectshavepartswithcomplexdependencies.Weneedtomodelthese.MRFs(andbeliefnets)modelcomplexdependenciesΨ(xi,xj)4142scene7\nNetworkjointprobabilityInordertouseMRFs:•Givenobservationsy,andtheparametersoftheMRF,howinferthehiddenvariables,x?1P(x,y)=∏Ψ(xi,xj)∏Φ(xi,yi)•HowlearntheparametersoftheMRF?Zi,jisceneScene-sceneImage-sceneimagecompatibilitycompatibilityfunctionfunctionneighboringlocalscenenodesobservations4344MRFinferenceClassesofgraphicalmodelsProbabilisticmodels•InferenceinMRF’s.Graphicalmodels–Bayesianinference–Gibbssampling,simulatedannealingDirectedUndirected–Iteratedcondtionalmodes(ICM)–VariationalmethodsBayesnetsMRFs–BeliefpropagationDBNs–Graphcuts4546References•Probabilisticinferenceingraphicalmodels–inHandbookofNeuralNetworksandBrainTheory,2002–MichaelI.Jordan–U.C.Berkeley•GraphicalModels–Chapter8ofPatternRecognitionandMachineLearning,2006–ChristopherM.Bishop–Microsoft•Graphicalmodelsoftwareformachinelearning–KevinMurphy–UniversityofBritishColumbia47488\nOutline时序状态空间模型下的贝叶斯估计•Introduction•Graphicalmodels•StatespacemodelandBayesianrule•Challengesinvisualmotionanalysis(VMA)?•HowtodoresearchonVMA?49505152时序状态空间模型下的贝叶斯估计ß贝叶斯估计假设:1)状态传递遵循Markov过程pp()xxxkk−−12,,,k?x0=()xxkk−12)当前观测量独立于过去时刻的状态量pA()ykkx,=p()ykkx则后验概率为pp()yxkk()xYkk−1p()xYkk=p()yYkk−1其中，先验概率为pp()xYkk−−11=∫()xxkkp()xYk−1k−1dxk−1似然函数为p()ykkxpp()yYkk−−11=∫()yxkkp()xYkkdxk53549\nOutline•Introduction•Graphicalmodels•StatespacemodelandBayesianrule•Challengesinvisualmotionanalysis(VMA)?•HowtodoresearchonVMA?5556ChallengesinmotionanalysisSummary•Non-rigidorarticulatednatureofobjects•视觉推理实例•Partialorfullobjectocclusion–Tracking,poseestimation•Complexobjectshapes•理论•Sceneilluminationchanges–Graphicalmodels,SSM,DBN,MRF•Realtimeprocessingrequirements•工具•Efficientmodeling&inference–DBNcode•High-dimensionalstatespace–Dimensionalreduction–Efficientsampling5758HowtodoresearchonVMA读文章的几个阶段•数据驱动，目标明确化、细节化•浏览最新文章–输入–PAMI,IJCV,ICCV,CVPR,ECCV,NIPS–T-IP,ICIP,ICPR–输出–T-RO,ICRA,IROS•从具体问题到数学问题的抽象–PhDthesis–已知•精读关键文章–求解–实现，并确定待突破点•高起点，博采众长•浏览相关文章•关键是实现•精读密切相关文章•新方法的提出、实现、比较596010\nQ?SeeyounextTuesday!6111