Usando MongoDB como storage de arquivos #1

julho 20th, 2011 by caferrari Leave a reply »

Estou envolvido em um projeto que vai necessitar armazenar muitos arquivos, e precisava de uma solução que fosse tolerante a falhas e escalasse muito bem, tanto verticalmente (Mais disco) quanto horizontalmente (Mais servidores).

Depois de olhar algumas soluções existentes como MogileFS, Cassandra, entre outros, eu gostei bastante do GridFS do MongoDB, pela simplicidade de configuração e principalmente pelo sistema de eleição de um novo servidor mestre caso o ativo venha a falhar em um determinado replicaSet.

Minha configuração inicial dessa estrutura consiste em dois servidores de Storage (Ubuntu Server 11.04 64bits rodando em VMs vmware em discos diferentes) e uma terceira maquina simples e remota que será um backup onde será configurado meu primeiro ReplicaSet. No servidor de aplicação vou ter o MongoDB rodando como arbitro (sem armazenar dados) apenas para determinar qual o servidor é o mestre caso algum venha a falhar.

Instalação:

Considerando que nos servidores foi instalado o ubuntu server, sem nada, vamos a configuração:

primeiro vamos adicionar a linha na configuração do APT para podermos baixar o pacote oficial do MongoDB, mantido pela 10-gen, crie o arquivo: /etc/apt/sources.list.d/mongo.list e adicione a linha:

deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen

depois de salvar o arquivo, adicione a chave do repositorio:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10

e finalmente instale o mongodb:

sudo aptitude update
sudo aptitude install mongodb-10gen

repita este processo em todas as maquinas de seu replicaSet.

Configuração do MongoDB

Primeiro vamos criar a pasta de armazenamento dos dados e definir as permissões:

sudo mkdir /dados
sudo chown mongodb:mongodb /dados/
sudo chmod 700 mongodb -Rf

Em todos os servidores vamos editar a configuração do MongoDB em /etc/mongodb.conf, aqui ficou assim:

dbpath=/dados/
logpath=/var/log/mongodb/mongodb.log
logappend=true
replSet = arquivo1
rest=true

* o parâmetro rest, faz com o que o servidor crie uma página para monitoramento dos status.

Depois de configurar todos os membros do replicaSet, reinicie o processo em todas as maquinas:

sudo service mongodb restart

Agora temos os servidores prontos para configurar a parte mais simples, o replicaset

Configuração do ReplicaSet

Para facilitar, configure nomes para seus membros do replicaSet no seu servidor de DNS interno ou, edite o arquivo /etc/hosts e adicione as maquinas que serão membro do replicaSet:

192.168.0.100      arquivo1a 
192.168.0.101      arquivo1b
192.168.0.102      arquivo1bkp

* Ajuste os IPs de acordo com sua rede.

Agora, em um servidor qualquer, acesse o console de administração do Mongo:

mongo admin

e crie a configuração do replicaSet:

conf = { _id : 'arquivo1',                    
... members : [
... {_id: 0, host: 'arquivo1a'}, 
... {_id: 1, host: 'arquivo1b'},
... {_id: 2, host: 'arquivo1bkp', priority: 0}
... ]}

* Note que o servidor de backup tem prioridade zero, isto faz com que este servidor nunca seja eleito como servidor Mestre.
confira se está tudo ok:

{
	"_id" : "arquivo1",
	"members" : [
		{
			"_id" : 0,
			"host" : "arquivo1a"
		},
		{
			"_id" : 1,
			"host" : "arquivo1b"
		},
		{
			"_id" : 2,
			"host" : "arquivo1bkp",
			"priority" : 0
		}
	]
}

se estiver ok, ainda no console do Mongo, inicialize o replicaset:

rs.initiate(conf)

em 10 segundos o seu replicaSet deverá estar rodando e pronto para receber dados, veja como está pelo navegador, acessando:

http://192.168.0.100:28017/_replSet

Você verá uma tela com o status de todos os membros do replicaSet:

Bom, é isso, no próximo post vou explicar como configurar o php e efetivamente armazenar arquivos no GridFS, Abraços

Advertisement

278 comments

  1. Une supplémentation en vitamines, minéraux et omégas 3 est une base pour
    apporter au corps l’ensemble des micronutriments conseillés.

  2. I visited ᴠarious webb sites eҳcept tɦe audio feature for audio songs existing at thiѕ webb ⲣage iss rᥱally marvelous.

    Hаve a look at myy blog post … addmefast points hack

  3. celulite disse:

    After going over a numbᥱr oof thе articles on your blog, Ⅰ really like yoսr
    technique оf blogging. I book-marked іt to my bookmark website list and wіll be checking back іn the near future.
    Plᥱase visit mу web site aѕ well and tеll me wһat you think.

    Alѕo vizit my web paցe: celulite

  4. fucking mom disse:

    Appreciating thee tіme and energy yoᥙ put іnto youur site and in depth іnformation you prеsᥱnt.
    It’s awesome to ϲome ɑcross a blog еvery оnce іn ɑ ѡhile that іsn’t tһe same old rehashed material.
    Excellent гead! I’ѵe bookmarked ʏour site and I’m adding үouг RSS feeds tߋ my Google account.

    Check oᥙt my webpage … fucking mom

  5. fireplaces 4k disse:

    Nice post. I was checking continuously this blog and I am impressed!
    Extremely helpful info specifically the last section :
    ) I care for such information much. I used to be looking for this certain information for a
    very lengthy time. Thanks and best of luck.

    My web-site fireplaces 4k

  6. La trottinette électrique légère et design de citybug avec commandes au guidon et roue gonflée.

  7. Following a rise in the vaгiеty of sgores in 2013, big bazaar telenor
    Bazar once more opened fօrty-three new stoгes in 2013,
    inclᥙdung thirty-seven within the Netherlands and six in Beⅼgium.

  8. Next, combine up a 5-gallon batch of water and Polished concrete floor finishes cleaner/degreaser in line with
    label directions (discovered at dwelling centers and hardware stores).

  9. They would have different sets of Indian bridal jewelry for each occasions custom-developed preferably by identified jewelry designers.

  10. Every weekend i used to visit this web site,
    for the reason that i want enjoyment, since this this website
    conations genuinely good funny stuff too.

  11. 13642 disse:

    Great post. I used to be checking constantly this blog and I
    am inspired! Extremely helpful info specifically the closing part :) I
    maintain such info much. I used to be looking for
    this certain information for a long time. Thanks and good luck.

  12. https://walidtourism.com

    شركات سياحية في تركيا,رحلات سياحية الى تركيا,برامج سياحية الى تركيا,رحلات داخلية في اسطنبول,تاجير سيارات مع سائق في تركيا,استئجار سيارات في اسطنبول مع سائق,رحلات شهر العسل في تركيا

  13. I’m no longer positive where you are getting your
    information, however good topic. I needs to spend some
    time finding out more or working out more. Thank you for magnificent information I was on the lookout for this info
    for my mission.

  14. http://www.almydannews.com

    الميدان , جريدة الميدان , موقع
    الميدان الاخبارى , موقع الميدان الالكترونى
    , الموقع الالكترونى لجريدة الميدان

  15. https://www.satellite-kuwait.net/

    فني ستلايت , فني هندي ستلايت , فني
    ستلايت الكويت , فني ستلايت بالكويت ,
    فني ستلايت حولي , فني ستلايت السالمية , فني ستلايت الفروانية , فني ستلايت العاصمة , فني ستلايت مبارك الكبير , فني ستلايت الجهراء , فني ستلايت الاحمدي , رقم فني ستلايت

  16. https://www.hakeemy.com

    حكيمي

    طبي ,دكتور, طب ,مستشغى ,مستشفيات, حجز موعد ,طبيب, طبيبة ,اطباء, داكترة ,معالجة ,شفاء ,حكيمي,زرع الشعر, طبيبي الخاص, طبيب
    العائلة, اسنان ,جوجل بلاي

  17. Our users send us feedback, the most popular working cheat code for EA SPORTS UFC® game was 1600 Gold :
    DyCB3ToE8oGyYil_fa”.

  18. あなたの自家用車が必ず欲しいといった中古車買取業者がいると、相場価格以上の見積もりが差しだされることも可能性としては低くはないので、
    競争式の入札で中古車一括査定サイト等をうまく活用していくことはとても重要なポイントだと思います。
    インターネットの一括車査定サービスによって、中古車の売却相場が相違します。相場価格が変わるわけは、
    そのサービス毎に、加入会社がいくらか代わるからです。
    中古車査定では、相当数のポイントがあって、それに従って弾き出された見積もり価格と、中古車買取ショップとのやり取りにより、
    買取金額が成り立っています。

  19. This is really attention-grabbing, You’re a very professional blogger.
    I’ve joined your rss feed and sit up for in quest
    of extra of your magnificent post. Additionally, I have shared your website
    in my social networks

  20. trade bearing disse:

    I have read so many articles or reviews about the blogger lovers however this article is in fact
    a nice article, keep it up.

  21. Excellent post. I was checking constantly this blog and I’m inspired!
    Very useful info particularly the remaining phase :) I handle such
    info much. I used to be looking for this certain information for a long time.
    Thanks and best of luck.

  22. Norine disse:

    That is properly above the nationwide salary average for accountants, which is around $sixty five,000.

  23. sophie evans disse:

    great ⲣoints altogether, youu simply ԝⲟn a new reader.
    What mіght yoս recommend in гegards tо your pkst tuat ʏoᥙ just made a fеw dayѕ ago?

    Any positive?

    Аlso visdit mу web site – sophie evans

  24. http://alsadanygroup.com/limousine

    ليموزين , ليموزين مطار القاهرة , ليموزين مصر, ليموزين الاسكندريه ,
    ليموزين الساحل الشمالى, ليموزين الغردقة ,
    ليموزين مطار برج العرب

  25. Ꮩery nice blog post. ӏ cerrainly apprecate tɦis website.
    Stick ԝith it!

    Herе iss my web page girls scissoring

  26. nude disse:

    Hello Dear, aree үou genuinely visiting thіѕ website daily, if ѕo
    afterward you wilkl absolսtely take nice knoѡ-how.

    Mʏ blog post nude

  27. sony earphone disse:

    Regardless of the reason why, writing thank you notes has been one habit that
    many of us has never gotten rid of through
    the years. Therefore, the music would not be as impressive to the listeners as the artists want them to be.
    After all it is your intent to say how thankful you are for the person you are giving it to, right.

  28. In Gallery 5, an installation is presented by the gallery by Kelley Johnson named Somewhere Between Here and
    There.” Here Is The performer’ sixth solo display together with
    the gallery.

  29. Ramon disse:

    What a material oof un-ambiguity and preserveness of valuable knowledgge concerning unexpected emotions.

  30. steepster.com disse:

    Unquestionably beliewve that which you said. Youur favorite reason seemed too be
    on the internet tthe easiest thing to be aware of. I say to you,
    I certainly gett annoyed while peopke consider worries that
    they plainly don’t know about. Youu managed to hit thhe nail upon the top and also defined out
    thee whole thing without having side-effects , people can take a signal.
    Will likely be back to get more. Thanks

  31. Angelica disse:

    Met specifieke zoekwoorden, LSI en SILO SEARCH ENGINE OPTIMISATION
    en de combinatie van longtail zoekwoorden kunt u de meeste en beste resultaten behalen in zoekmachines.

  32. Aucun risque de ressembler à un Malabar.

  33. أجهزة منزلية,ألكترونيات,الصحة
    و الجمال,المنزل و معدات المطبخ,تابليت,مستلزمات الأطفال,موبيلات,اكسسوارات

    https://kzashop.com/

  34. Sofia disse:

    Lawns that aren’t getting sufficient nitrogen (the important
    thing element of garden fertilizer) will begin to
    alter to light green after which yellow.

  35. It’s in point of fact a nice and useful piece of info.
    I am happy that you just shared this useful info with us.
    Please keep us informed like this. Thank you for sharing.

  36. Install games on your phone, get gift cards, and all kinds of stuff for
    absolutely free! You’ve got to check this out!

  37. Obtain the class content in the conditioning corporation and then routine to
    take a test at a screening centre close by. This method may take about six
    to eight days depending on the personis expertise and knowledge, as well as the arrangement of the exam.

  38. I was very pleased to find this page. I wanted to thank you for ones
    time for this wonderful read!! I definitely savored every bit
    of it and i also have you book-marked to look at new information on your web site.

  39. Yes! Finally someone writes about Usaha makanan.

  40. Fantastic beat ! I wish to apprentice at the same time as
    you amend your website, how can i subscribe for a weblog site?
    The account helped me a applicable deal. I had been a little bit familiar
    of this your broadcast provided shiny clear concept

  41. For instance, if you must cut your 1-hour training session brief
    one day, your trainer can complete a routine with you that burns the identical number of
    energy and provides the same physical advantages in half the time.

  42. ex4 to mq4 disse:

    Thank you for another informative blog. The place else
    may just I am getting that type of info written in such an ideal approach?
    I have a project that I am simply now working
    on, and I have been at the look out for such info.

    My homepage ex4 to mq4

  43. Just desire to say your article is as surprising.

    The clarity in your submit is just spectacular and that i can assume you’re knowledgeable on this
    subject. Fine along with your permission allow me to
    snatch your RSS feed to keep up to date with forthcoming post.
    Thanks one million and please carry on the rewarding work.

  44. An outstanding share! I have just forwarded this onto a friend
    who had been conducting a little homework on this. And he
    actually bought me dinner because I stumbled upon it for
    him… lol. So allow me to reword this…. Thank YOU for the meal!!
    But yeah, thanx for spending some time to talk about this subject here
    on your web page.

  45. Erectile Dysfunction

    Vascular Illness: Often known as arteriosclerosis, vascular
    disease is a condition in which the arteries harden and
    narrow, resulting in a discount in blood circulate
    all through the body. This examine profiles the underlying threat
    for ailments associated with ED in a cohort of Canadian males with gentle ED who
    participated in a randomized DBPC trial of sildenafil versus placebo and compares the
    results with the chance in a database of men representing the general ED
    scientific trial population.

    The motion of Panax ginseng seems to be handiest for these with excessive lipids in their blood
    and metabolic syndrome This herb is thought to have anti-inflammatory action, enhance lung perform, and improve blood stream in different diseases — all characteristics
    which will scale back ED.

    Dysfunctions were transitory and experienced by the vast majority of folks, dysfunctions bounded male major or secondary impotence ,
    premature ejaculation , ejaculatory incompetence ; feminine major orgasmic dysfunction and situational orgasmic dysfunction; ache throughout intercourse ( dyspareunia ) and
    vaginismus In response to Masters and Johnson sexual arousal and climax are a traditional physiological
    course of of every functionally intact grownup, but despite being autonomic it can be inhibited.

    Imply BMI was related (29 ± 5 kg/m2, range, 20-forty eight kg/m2 vs 28 ± 5 kg/m2, vary, 11-64 kg/m2), but extra males in the
    gentle ED population had been obese as defined by the World Health Group 28 (BMI ≥ 30 kg/m2;
    34% vs 24%).

    Males with ED may additionally expertise anxiousness or
    depression 56 Erectile dysfunction is widespread in men with lower urinary tract signs associated
    to BPH. If you’re ready to start seeing improvements, contemplate making an attempt
    among the natural treatments for impotence explained below.

  46. .WNaf-_mGPDc disse:

    It’s possible you’ll know that promoting for creditors shouldn’t be
    required in every estate; it’s one thing
    that each executor should decide on. Part of the explanation it is finished
    is to guard the executor from the creditors.

  47. FSM-15S disse:

    Attractive section of content. I simply stumbled upon your
    web site and in accession capital to claim that I acquire in fact enjoyed account
    your weblog posts. Any way I’ll be subscribing to
    your feeds and even I fulfillment you get admission to consistently quickly.

  48. I got this site from my buddy who shared with me regarding this site and now this time I am visiting
    this web site and reading very informative content at this place.

  49. Hi there I am so thrilled I found your blog page, I really
    found you by mistake, while I was researching on Bing for something else, Anyways I am here now and would just like to
    say kudos for a tremendous post and a all round interesting blog (I also love the theme/design), I don’t have time to go
    through it all at the minute but I have book-marked it and also added your RSS feeds, so when I have time I will be back to read a
    lot more, Please do keep up the great work.

  50. First off I would like to say fantastic blog! I had a quick question in which I’d like to ask if you do not mind.
    I was interested to know how you center yourself and clear your head before writing.
    I’ve had a difficult time clearing my mind in getting
    my thoughts out there. I do enjoy writing but it just seems like the first 10 to 15 minutes are usually
    lost just trying to figure out how to begin. Any recommendations or hints?

    Cheers!

Trackbacks /
Pingbacks

Deixe uma resposta