opensubscriber
   Find in this group all groups
 
Unknown more information…

f : fmetalk@googlegroups.com 24 January 2011 • 12:48PM -0500

[fme] Re: Detecting and removing outliers
by Darwindpi

REPLY TO AUTHOR
 
REPLY TO GROUP




Daniel,

Several hundred, that's different.

I'm going to wing it here.

1. If the outiers are a long way away from the "good" data you could
use a bounding box replacer or convex hull generator on the entire
data set.  Then use a bufferer with a negative buffer to create a new
polygon which is smaller by some amount than the first one.  Use this
as a clipper.

2. If the outiers differ from the good data by some attribute,
geometry or both you could test for them.

3. Using the bounding box or convex hull create a centerpoint
replacer, or if the distribution is uneven, a centerofgravity
replacer, to create a centroid. Then there are two possibilities, you
could try the neighborfinder, use the centroid as the BASE, use the
neighbor list to list all neighbors and their distances.  Then use the
statistics generator to determin the STDDEV.  Or you could extract the
coordintates (assuming the data is points) and using the expression
evaluator and the hypotenuese formula to calculate distances to the
centroid. I've never much liked the expresson evaluator.  I find the
syntax hard to get right and use it only when I've exhausted all other
possibilities.

I haven't tested any of this, it could all be bogus.

Russell



On Jan 24, 11:30 am, Daniel <dtm...@gmai...> wrote:
> Hi Darwindpi,
>
> I was kinda hoping for a more automated way of doing it. The method
> you're suggesting requires me to define the bounding box for each DGN
> file, but I've got several hundred of these to process, and for future
> use I'd like to just run a fme script to get my result.
>
> I thought that if there was a reliable way to define the centroid (ie
> removing outliers), then you could get each feature's standard
> deviation from the centroid and remove them if they are beyond a
> certain amount. Is it possible to do this in a workbench?
> (Of course I'm also open to other suggestions)
>
> Thanks,
>
> Daniel
>
> On Jan 24, 7:46 am, Darwindpi <russell.mil...@nt.g...> wrote:
>
>
>
> > Dan,
> > I work with DGN files all the time and occasionally get the same
> > problem.  I usually delete the data from the dgn by using either a
> > void-clip fence or a select-by-attribute set to "exclude" and delete.
> > Whatever works.
> > If I wanted to do the same in workbench my first try would be to
> > define a bounding box around the data I want and use the clipper
> > transformer to remove the "clipped outside".
>
> > Hope this helps,
>
> > Russell
>
> > On Jan 19, 12:23 pm, Daniel <dtm...@gmai...> wrote:
>
> > > Hi All,
>
> > > This is my first post in the FME forums, so go easy on me ^_^.
>
> > > My scenario is that I'm trying to import DGN data into a postgis table
> > > so that I can do some post processing. While doing this, I realize
> > > that the data sometimes have outlying data (i'm talking like points
> > > off the planet) and this not only impacts some of my calculations
> > > (such as centroid), it also kinda ticks me off...
>
> > > So, is there a nice FME transform that can detect and remove outliers?
>
> > > Thanks,
>
> > > Dan- Hide quoted text -
>
> - Show quoted text -

--
Get all the FME resources you need at http://www.fmeusercentral.com

You received this message because you are subscribed to the Google Groups "FME Talk" group.

To post to this group, send email to fmetalk@goog...
To unsubscribe from this group, send email to fmetalk-unsubscribe@goog...

Bookmark with:

Delicious   Digg   reddit   Facebook   StumbleUpon

Related Messages

opensubscriber is not affiliated with the authors of this message nor responsible for its content.