Giter VIP home page Giter VIP logo

prebid-server's Introduction

Build Go Report Card Go Version



Prebid Server Logo


Prebid Server is an open-source solution for running real-time advertising auctions in the cloud. This project is part of the Prebid ecosystem, seamlessly integrating with Prebid.js and the Prebid Mobile SDKs to deliver world-class header bidding for any ad format and for any type of digital media.

Getting Started

Configuring

When hosting Prebid Server or developing locally, you must set a default GDPR value. This configuration determines whether GDPR is enabled when no regulatory signal is available in the request, where a value of "0" disables it by default and a value of "1" enables it. This is required as there is no consensus on a good default.

Refer to the configuration guide for additional information and a list of available configuration options.

Hosting Prebid Server

Note

Please consider registering as a Prebid Server host to join the mailing list for updates and feedback.

The quickest way to host Prebid Server is to deploy our official Docker image. If you're hosting the container with Kubernetes, you can configure Prebid Server with environment variables using a pod file or using a config map. Alternatively, you can use a configuration file embedded in a config map which Prebid Server will read from at the path /etc/config.

For deploying a fork, you can create a custom Docker container using the command:

docker build --platform linux/amd64 -t prebid-server .

or compile a standalone binary using the command:

go build .

Ensure that you deploy the /static directory, as Prebid Server requires those files at startup.

Developing

Prebid Server requires Go version 1.21 or newer. You can develop on any operating system that Go supports; however, please note that our helper scripts are written in bash.

  1. Clone The Repository
git clone [email protected]:prebid/prebid-server.git
cd prebid-server
  1. Download Dependencies
go mod download
  1. Verify Tests Pass
./validate.sh
  1. Run The Server
go run .

By default, Prebid Server will attach to port 8000. To confirm the server is running, visit http://localhost:8000/ in your web browser.

Code Style

To maintain consistency in the project's code, please:

  • Follow the recommendations set by Effective Go. This article provides a comprehensive guide on how to write idiomatic Go code, covering topics such as naming and formatting. Many IDEs will automatically format your code upon save. If you need to manaully format your code, either run the bash script or execute the make step:

    ./scripts/format.sh -f true
    
    make format
    
  • Prefer small functions with descriptive names instead of complex functions with comments. This approach helps make the code more readable, maintainable, and testable.

  • Do not discard errors. You should implement appropriate error handling, such as gracefully falling back to a default behavior or bubbling up an error.

IDE Recommendation

An option for developing Prebid Server in a reproducible environment isolated from your host OS is using Visual Studio Code with Remote Container Setup. This is a recommendation, not a requirement. This approach is especially useful if you are developing on Windows, since the Remote Container runs within WSL providing you with the capability to execute bash scripts.

Importing Prebid Server

Prebid Server is not currently intended to be imported by other projects. Go Modules is used to manage dependencies, which also makes it possible to import Prebid Server packages. This is not supported. We offer no guarantees regarding the stability of packages and do not adhere to semantic versioning guidelines.

Contributing

Important

All contributions must follow the Prebid Code of Conduct and the Prebid Module Rules.

Bid Adapter

Bid Adapters transform OpenRTB requests and responses for communicating with a bidding server. This may be as simple as a passthrough or as complex as mapping to a custom data model. We invite you to contribute an adapter for your company. Consult our guide on building a bid adapter for more information.

Analytics Module

Analytics Modules enable business intelligence tools to collect data from Prebid Server to provide publishers and hosts with valuable insights into their header bidding traffic. We welcome you to contribute a module for your platform. Refer to our guide on building an analytics module for further information.

Auction Module

Auction Modules allow hosts to extend the behavior of Prebid Server at specfic spots in the auction pipeline using existing modules or by developing custom functionality. Auction Modules may provide creative validation, traffic optimization, and real time data services among many other potential uses. We welcome vendors and community members to contribute modules that publishers and hosts may find useful. Consult our guide on building an auction module for more information.

Feature

We welcome everyone to contribute to this project by implementing a specification or by proposing a new feature. Please review the prioritized project board, where you can select an issue labeled "Ready For Dev". To avoid redundant effort, kindly leave a comment on the issue stating your intention to take it on. To propose a feature, open a new issue with as much detail as possible for consideration by the Prebid Server Committee.

Bug Fix

Bug reports may be submitted by opening a new issue and describing the error in detail with the steps to reproduce and example data. A member of the core development team will validate the bug and discuss next steps. You're encouraged to open an exploratory draft pull request to either demonstrate the bug by adding a test or offering a potential fix. The quickest way to start developing Prebid Server in a reproducible environment isolated from your host OS is by using Visual Studio Code with Remote Container Setup.

Learning Materials

To understand more about how Prebid Server in Go works and quickly spins up sample instances, refer to the sample folder which describes various structured and integrated examples. The examples are designed to run on any platform that supports docker container.

prebid-server's People

Contributors

aadeshp avatar alexbvolcy avatar benjaminch avatar bokelley avatar bretg avatar bsardo avatar camrice avatar ccorbo avatar cirla avatar ckbo3hrk avatar dbemiller avatar dmitris avatar gargcreation1992 avatar genegenie avatar guscarreon avatar hhhjort avatar laurb9 avatar mansinahar avatar mikael-lundin avatar muncha avatar nhedley avatar onkarvhanumante avatar pm-nilesh-chate avatar pubmatic-openwrap avatar serhiinahornyi avatar smithaammassamveettil avatar sonali-more-xandr avatar syntaxnode avatar themediagrid avatar veronikasolovei9 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

prebid-server's Issues

inherit hosting partner cookie matching table

Currently, PBS supports inheriting uuid cookie from domain hosting the PBS solution.

That means, when AppNexus host PBS, it automatically gets the UUID2 cookie value from the user and can use it right away. Same, when Rubicon host PBS, they can use this feature to automatically inherit their user id Rubicon cookie.

However, the PBS solution doesn't inherit from the hosting partner existing cookie matching table.
Meaning, in case AppNexus host PBS, we would inherit from the UUID2 cookie, but even if AN already did the matching for that UUID2 with sevral SSP partners, PBS will have to do this matching on his own.

Would be great if we could provide a url in the config, so that when PBS inherit from the hosting partner UUID, it can call that url with the UUID, and it returns the existing mapping table from the hosting solution, so PBS doesn't need to do his own sync.

Prebid Server is responding no_bid.

We have two ad unit slots, one having valid AppNexus placement id, another having invalid AppNexus placement id. When a call goes to Prebid Server it returns no_bid for all ad units.

Eg. Consider adunit1 and adunit2, valid Appnexus placement id APNX1 (5823300), invalid Appnexus placement id APNX2 (5823302).

If we call like
AdUnit1 - APNX1
AdUnit2 - APNX1

Prebid server responded with valid bids.

If we call like
AdUnit1 - APNX1
AdUnit2 - APNX2

Prebid server responded with invalid bids.

What is Prebid servers internal implementation? If one platform does not give bid then will it return no bid?

rubicon produces error on non-standard size

We have an ad unit with multiple sizes: 300x250 and 300x610. We have some demand for 300x610 in appnexus, but most of the time this ad unit works as normal 300x250. It works with pbjs, but inside the prebid server rubicon adapter produces an error that says that 300x610 is not supported.

As all bidders get the same list of sizes, it's not possible to disable 300x610 just for rubicon or use it just for appnexus. So it would be nice if rubicon adapter could just filter-out those sizes that it doesn't support.

Move Adapters into separate packages

Most external contributions to Prebid.JS are new bidder adapters... and I expect Prebid-Server to be very similar.

Now that #144 is in, the CI will guarantee minimum code coverage standards in each package.

With all the adapters in the same package, new adapters with weak tests can slip in as long as other adapters have good coverage.

We should move each adapter into its own package, so that the CI holds each adapter to the same standard. It's also good for code organization, since adapters probably shouldn't have access to each others' private structs and methods.

I propose we move each adapter into its own package, like adapters/appnexus, adapters/rubicon, etc.

OpenRTB support: Rubicon adapter

Now that #196 has landed, the /openrtb2/auction endpoint is (technically) live. This deprecates /auction, and moves the project towards #150.

The next step is to update existing adapters to support it. Documentation can be found here. I'm not sure whether it's easier for you to create a new struct or add functions to your existing one. Both are equally sensible as far as I'm concerned.

@bretg @DucChau This issue is for updating the Rubicon adapter. Please let me know if you have questions (or, if you figure things out, submit a PR to the docs to clarify things)

Mismatched type for Prebid Server

Configuration (sample):

pbjs.setConfig({
	bidderTimeout: PREBID_TIMEOUT,
	enableSendAllBids: true,
	s2sConfig: {
		accountId: 'e7445f8d-ee19-4a94-a234-a271cb3e6bc0',
		enabled: true,
		bidders: ['facebook', 'pulsepoint', 'rubicon'],
		timeout: PREBID_TIMEOUT
	} 
});

pbjs.addAdUnits([{
	code: "div-gpt-ad-1418309082456-0",
	sizes: [
		[728, 90],
		[970, 250]
	],
	bids: [{
		bidder: "rubicon",
		params: {
			accountId: "14346",
			siteId: "154972",
			zoneId: "738672",
			sizes: "[2,57]"
		}
	}]
}]);

Result with ?pbjs_debug=true - no call is made:

MESSAGE: CALLING S2S HEADER BIDDERS ==== facebook,pulsepoint,rubicon
MESSAGE: Mismatched type for Prebid Server : rubicon : accountId. Required Type:function () { [native code] }
MESSAGE: Mismatched type for Prebid Server : rubicon : siteId. Required Type:function () { [native code] }
MESSAGE: Mismatched type for Prebid Server : rubicon : zoneId. Required Type:function () { [native code] }

Help?

P.S. prebid.js build 0.32.0.

Add APIs for bidder params

In Prebid.js, someone requested a bidder params API.

It's a good idea here, as well. And since this project runs a server... it's much easier :).

I propose adding endpoint(s) which return json-schema for all the known bidders and bidder params. This could be used to help generate API-driven documentation, config management UIs, etc.

OpenRTB support: Lifestreet adapter

Now that #196 has landed, the /openrtb2/auction endpoint is (technically) live... with one bidder. This deprecates /auction, and moves the project towards #150.

The next step is to update existing adapters to support it. Documentation can be found here.

@Lifestreet: This issue is for making a Lifestreet Bidder, or updating the existing one, to support OpenRTB.

appnexus does not support randomKey parameter

Appnexus bidder allows to pass random key-value pairs to enhance buy-side targeting: http://prebid.org/dev-docs/bidders.html#appnexus (see randomKey).

Here is the bids section from request payload that was sent to prebid-server:

"bids": [{
    "bidder": "appnexus",
    "params": {
       "placementId": 10876685,
    "is_refresh": "false"
  },
  "bid_id": "1555b68f1fadef"
}],

And here is request body that was sent by appnexus server side adapter as part of rtb request:

{
    "id": "f6abc016-4bdd-4c37-9bdd-653abda9815a",
    "imp": [{
        "id": "ad_cs_2010299_728_90",
        "banner": {
            "w": 728,
            "h": 90,
            "format": [{
                "w": 728,
                "h": 90
            }, {
                "w": 970,
                "h": 250
            }]
        },
        "secure": 1,
        "ext": {
            "appnexus": {
                "placement_id": 10876685
            }
        }
    }],
    "site": {
        "domain": "%removed%",
        "page": "%removed%"
    },
    "device": {
        "ua": "%removed%",
        "ip": "%removed%"
    },
    "user": {
        "id": "9141190353768996862",
        "buyeruid": "9141190353768996862"
    },
    "at": 1,
    "tmax": 1000,
    "source": {
        "fd": 1,
        "tid": "f6abc016-4bdd-4c37-9bdd-653abda9815a"
    }
}

As you can see custom "is_refresh" parameter was not passed in rtb request.

video type supply returns "banner" type

Currently if you request a video adunit and a bid responds, it shows as "banner".

Example curl:

curl 'http://prebid.adnxs.com/pbs/v1/auction' -H 'Pragma: no-cache' -H 'Origin: http://ap.localhost:9999' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36' -H 'Content-Type: text/plain' -H 'Accept: */*' -H 'Cache-Control: no-cache' -H 'Referer: http://ap.localhost:9999/integrationExamples/s2s_instream_video.html' -H 'Connection: keep-alive' --data-binary '{"account_id":"c9d412ee-3cc6-4b66-9326-9f49d528f13e","tid":"f7170633-561f-4822-837c-703477db20c0","max_bids":1,"timeout_millis":1000,"secure":0,"url":"http://ap.localhost:9999/integrationExamples/s2s_instream_video.html","prebid_version":"0.30.1","ad_units":[{"code":"/19968336/header-bid-tag-0","sizes":[{"w":640,"h":480}],"bids":[{"bidder":"appnexus","params":{"placementId":9333431},"bid_id":"1e299f66fadca2"}],"transactionId":"6ef2b100-7477-4e0d-bee8-8e4df9850329","video":{"mimes":["video/mp4"],"playback_method":2,"protocols":[1,2,3,4,5,6,7,8]},"media_types":["video"]}],"is_debug":true}' --compressed

Response:

{
	"tid": "f7170633-561f-4822-837c-703477db20c0",
	"status": "no_cookie",
	"bidder_status": [{
		"bidder": "appnexus",
		"response_time_ms": 77,
		"num_bids": 1
	}],
	"bids": [{
		"bid_id": "1e299f66fadca2",
		"code": "/19968336/header-bid-tag-0",
		"creative_id": "53228341",
		"media_type": "banner",
		"bidder": "appnexus",
		"price": 5,
		"width": 1,
		"height": 1,
		"response_time_ms": 77
	}]
}

Experiment with fasthttp library

The fasthttp project project shows much better performance than Go core's net/http library.

It looks like there are some downsides too, though. The context type isn't the same as the one in Go's core, which means that it may be hard to use alongside other libraries. We would also lose native "graceful shutdown" support.

On the other hand, the benchmarks are awesome.

Since prebid-server deploys are expected to serve very high traffic across the internet, it'd be nice to get a more well-defined analysis of the tradeoffs between net/http and valyala/fasthttp. If they look reasonable, we may as well use them.

I cant make the prebid-server to work.

I have created two examples: one that makes header bidding without prebid-server and a second one with it.
The first test without prebid-server works fine, but the second one doesn't.

At the first call of the prebid-server auction url the response sends an url for user sync but prebid.js doesn't make the sync so i manually called the url returned by prebid-server/auction.

After i did this i receive from prebid-server auction this response

{"tid":"c730e370-277c-4cb3-bd4b-ee44fed1d82e","status":"OK","bidder_status":[{"bidder":"appnexus","response_time_ms":35,"no_bid":true}]}

This is the prebid-server config added at the second example:
pbjs.setS2SConfig({
accountId: '2d749ec2-6a1d-4205-bf6b-5dd8d41386f9',
enabled: true,
bidders: ['appnexus'],
timeout: 1000,
endpoint: 'http://devprebid.elasticad.net/auction'
});

Another thing i saw is that you have to add '/auction' to the endpoint in order to work. i'm not sure that this is intended or not.

I don't know if there is something wrong with the prebid-server or with my prebid.js configuration that the prebid server example doesn't work as expected.

Without PrebidServer
http://jsfiddle.net/photonx/ngdguvg8/

With PrebidServer
http://jsfiddle.net/photonx/L1n27nwx/

Pubmatic usersync doesn't work on http website

I run my test page on http, prebid server endpoint is http too. I send bid request and receive usersync field in response

{
    "bidder": "pubmatic",
    "no_cookie": true,
    "usersync": {
        "url": "https://ads.pubmatic.com/AdServer/js/user_sync.html?predirect=http://localhost:8000/setuid?bidder=pubmatic&uid=",
        "type": "iframe"
    }
}

Usersync url is https, and when https iframe tries to redirect back to http page I get Mixed Content error. The issue can be fixed it seems by removing protocol from usersyncURL

In pubmatic.go line 145:
usersyncURL := "//ads.pubmatic.com/AdServer/js/user_sync.html?predirect="

Create Auction endpoints which are OpenRTB-compliant

The /auction endpoint accepts JSON in a custom data structure.

But usually, it's a good idea to follow standards unless there's really good reason to do otherwise. There's the current OpenRTB 2.5 spec, as well as an OpenRTB 3.0 one coming soon-ish.

Would anyone object if we versioned the endpoint and began to follow the standard?

I receive this error on bid request

After i rebuild prebid.js with adapters [appnexus,prebidServer] adapters and added the S2SConfig object, i receive this error on bid request. I used the last versions from GIT for both prebid.js and prebid-server.
"ERROR: Error processing command :Cannot read property 'setConfig' of undefined" ""

OpenRTB support: Pubmatic adapter

Now that #196 has landed, the /openrtb2/auction endpoint is (technically) live... with one bidder. This deprecates /auction, and moves the project towards #150.

The next step is to update existing adapters to support it. Documentation can be found here.

@PubMatic-OpenWrap: This issue is for making a Pubmatic Bidder, or updating the existing one, to support OpenRTB.

OpenRTB Support: Sample Bidder Requests

Background

There are a few issues which I believe share a common core dependency.

Adapter race conditions

These are described in more detail in #141. Despite my best efforts, I was not able to solve it with a good OpenRTB Bidder API.

User-facing documentation

In Prebid.js 1.0, bidders are required to submit a markdown file with sample bid params. The goal is to define some "hello world" documentation for that bidder, and prove that the bidder works when the team tests PR changes.

There are some issues here which I think we can improve on:

  • The markdown files aren't machine-readable, so they may fall out of date at any time.
  • The team still has to manually test them every time an adapter PR is submitted, which slows PR reviews.

Unit test boilerplate

Each Bidder has very few responsibilities. Given an OpenRTB Request, make some HTTP Requests. Given some HTTP Responses, make some Bids.

Let's face it... Go is not (at least yet) a mainstream language. Unlike javascript, many buyers may not even have any engineers who know how to write it comfortably.

My theory is: the easier a Bidder is to write, the more Bidder submissions we'll get. We should do anything we can to make things as easy on new submissions as possible.

Proposal

I propose that we require Bidders to submit "test case data" in a machine-readable format. For a rough example:

{
  "bidRequest": { ... a sample OpenRTB Request ... },
  "httpCalls": [
    {
      "request": { ... http request data ... },
      "response": { ... http response data ... }
     },
     ... more http calls allowed here ...
  ],
  "bids": [
    ... OpenRTB Bids here ...
  ]
}

Purpose

This would contribute towards solutions for all of the problems listed above. In the future, we would use these to:

  • Collect imp values from the bidRequest, use the httpCalls to make a mock server, and run a massive integration test under the race detector

  • Create a docs page which is kept up-to-date automatically. The bidRequest can be parsed to get sample bidder params, and the httpCalls contains request/response data which we can validate in real time by making a request to their server. The docs page could mark each bidder as "working" or "may be broken" depending on if the request & response data matches

  • Centralize the adapter test code in a single test file. New adapters would be required to submit "test case data" (until they pass the code coverage threshold), but would not need to write any unit tests in Go.

Some Dangers

Certain parts of the OpenRTB spec are non-deterministic. For example, bid.id is randomly generated to assist with tracking & logging.

I'm not yet sure whether the best solution here is to allow for a flexible "expectations" format (for example, make bids[i].id into a regex), or simply omit those fields from any validation.

prebid.adnxs.com is not responding

Not sure if I should report this issue here, but it seems that prebid-server is down:

$ ping prebid.adnxs.com
PING prebid-ssl.northeurope.cloudapp.azure.com (13.74.186.28) 56(84) bytes of data.

--- prebid-ssl.northeurope.cloudapp.azure.com ping statistics ---
18 packets transmitted, 0 received, 100% packet loss, time 16999ms

Add video support to prebid server

  1. Prebid server will support mixed media types within one ad unit.
  2. PBS Request will be augmented in the following way:
    a. PBSAdUnit structure will have MediaTypes field added. The field is to be an array of enumerated type with the two values: “banner” and “video”. The array can have 1 or 2 elements of the type, repeating elements ignored. (So, it can be [“banner”], [“video”] or [“banner”, “video”]). For backward compatibility if the field is absent, it will be treated as [“banner”].
    b. A new filed Video will be added to PBSAdUnit. It will be either a reference to PBSVideo data structure, which will hold the parameters for Video (TBD) or NIL.
  3. The routine which converts PBSRequest into a generic OpenRTB request will be augmented in the following way:
    a. A new Boolean parameter “singleMediaTypeImp” will be added to the signature. If the parameter is true, for each media type (“banner” or “video”) a separate “Imp” element will be generated. Otherwise the Imp element will have data for both media types
    b. The routine will check submitted data to match the required media types; in case of mismatch (“video” media type is present, but the Video data structure is missing) the request should fail.
    c. The routine will add Video data to OpenRTB request as specified above.
  4. If necessary, PBSResponse will be adjusted to include video-related data.

Question About SSL

As a new adapter in prebid-server, what is the expected method for a publisher to request secure creatives?

Currently, Sovrn's bid endpoints use the request protocol to determine if only secure banners should be returned.

When taking a look at prebid-server (and the parameters passed into the adapters for requests), it doesn't seem that the adapters will have access to info (protocol used) about the original network request to /auction.

So, is the expectation that, in order to request only secure creatives, that a flag is set within the bidder json?

Rubicon calls setuid on prebid.adnxs.com event when endpoint is different

I get the following response from my prebid server

{
"bidder": "rubicon",
"no_cookie": true,
"usersync": {
"url": "https://pixel.rubiconproject.com/exchange/sync.php?p=prebid",
"type": "redirect"
}
}

Usersync url says nothing about the callback domain. And when prebid calls that usersync url it always calls https://prebid.adnxs.com/pbs/v1/setuid?bidder=rubicon&uid= even when I run prebid server locally and changed the endpoint accordingly. I guess adnxs.com url is hardcoded on rubicon's side.

Problem installing on FreeBSD

Hello.

I have a problem when i try to build in FreeBSD. 

vendor/github.com/cloudfoundry/gosigar/concrete_sigar.go:20: cpuUsage.Get undefined (type Cpu has no field or method Get)vendor/github.com/cloudfoundry/gosigar/concrete_sigar.go:30: cpuUsage.Get undefined (type Cpu has no field or method Get)vendor/github.com/cloudfoundry/gosigar/concrete_sigar.go:49: l.Get undefined (type LoadAverage has no field or method Get)vendor/github.com/cloudfoundry/gosigar/concrete_sigar.go:55: m.Get undefined (type Mem has no field or method Get)vendor/github.com/cloudfoundry/gosigar/concrete_sigar.go:61: s.Get undefined (type Swap has no field or method Get)vendor/github.com/cloudfoundry/gosigar/sigar_unix.go:23: cannot use stat.Ffree (type int64) as type uint64 in assignment

Prebid Server calls Cache unnecessarily

Right now, PBS sends a request to the Cache whenever cache_markup is true. If there are no bids, it still makes an empty request to the Cache.

There's no value in this... it just hurts performance. We should avoid this call.

OpenRTB support: Pulsepoint adapter

Now that #196 has landed, the /openrtb2/auction endpoint is (technically) live... with one bidder. This deprecates /auction, and moves the project towards #150.

The next step is to update existing adapters to support it. Documentation can be found here.

This issue is for making a Pulsepoint Bidder, or updating the existing one, to support OpenRTB.

OpenRTB support: Index adapter

Now that #196 has landed, the /openrtb2/auction endpoint is (technically) live... with one bidder. This deprecates /auction, and moves the project towards #150.

The next step is to update existing adapters to support it. Documentation can be found here.

This issue is for making or updating the Index adapter to support OpenRTB.

Rubicon responds with js as creative code. Prebid.js expects html/url

I've got the following response from prebid-server.

"bids": [{
"bid_id": "26f3cbd58fb4bb",
"code": "ad_cs_9493688_728_90",
"creative_id": "3563776",
"bidder": "rubicon",
"price": 0.230999,
"adm": "rubicon_cb = Math.random(); rubicon_rurl = document.referrer; if(top.location==document.location){rubicon_rurl = document.location;} rubicon_rurl = escape(rubicon_rurl);\nwindow.rubicon_ad = "3450174" + "." + "js"; window.rubicon_creative = "3563776" + "." + "js"; document.write("\n\n"); rubicon_tag_code = "%3cscript%20type=%22text/javascript%22%20src=%22https://pr.ybp.yahoo.com/ab/secure/true/imp/NA0gz5xIV5RA1_hBmpOHaeht06DqO-3uwFWAbdp6rlwC9DvRKwyDZAkacIV-Gr3FREiHzA0Deca3KihUQcVpKC5wSh9AvXjVGSMaGThhWebnTtt-Frxppd_a2uGfdUeXQUdZbErM30TPmhd9UI61ebzMhAm6odX9h_g5QVNgkBBRtKUjoMo_uFkN4dxtywDBTJwrfh0GG0Cg9551lxpMCf4qOlAV5ZoZ42Acw8riCArb5WXgtDdaqDJGT-GO4yYHP7nZltNXxAOhnQ5wSkWhIOOzs9dM3YegvvS6F1GFNsnMcYHzUg14hHbOSPKjWJIYJ0gJ3eMh13nXFEJvwz2EyZWPXeuvSCM3faZOidXMl-FIGYp1Z2wLwlx_sjT8_bw_K-wQvevLq64eaT8hzID8d6_nq-iO7DcQLz0qb8z46CcKoU06M1M6EiWRW0MF7X84OUmrOBQKl2gyvW1BXVqBmWgWDWzj3ZbNkjJ_ImYJeHg7SmuR2DkhskenWuGOB0CboUKFxHX-jSe0Ers1yILa_JZ1-lHpw3_nkA35Pu6pX1T1vPh2Nu0M3mZFwOGRVKuz1yBT0-6VX1Qxw2z4MEaEOJFH1BtwCKXC1hPRw3hRIG7TRdoD59ZshqtFavOwPQvrGkdkXHPwPFzqsfDqsXgXQ2KtjYmjgK2AOkUpjS4KhkH18haS1X3ImsgiucoqgzEB1BwyAiNpRa4tACwBWeWDvzAQQYFPPGOCh1m6wyF4JkzIwgzRKoVk7yQFlxhUhFiKtkOO3YliWd2OSLk6CqeogbQS_0I7HlqbUWza7xKSdnHkQZT5QpUoQAHu8vHmF57zKGbqDPX2EmjDWsbmbX39EwL-kik9pNetRwv84roDHa2E95f0yM7wKS_PhO-P55aoADSveP_qYLf3gYucsGfc3NIlwxOY2sAMCKPzcnmIA3k1Krbr3J8HUelz3AdTupld6DU8C_vRlHUgMOB652eryjHzpt4DzcFI/wp/E6103826DDE957DB%22%3e%3c/script%3e"; rubicon_tag_code = rubicon_tag_code.replace(/##RUBICON_CB##/g,rubicon_cb); document.write(unescape(rubicon_tag_code)); document.write("<div style=\"height:0px;width:0px;overflow:hidden\"><script>(function(){var proto=\'https:\';try{proto=window.top.location.protocol;}catch(e){proto=(document.referrer===\'\'?\'https:\':document.referrer.split(\'//\')[0]);}var server=\"http://tap2-cdn.rubiconproject.com\";if(proto==\"https:\")server=\"https://tap-secure.rubiconproject.com\";document.write(\'<iframe src=\"\'+server+\'/partner/scripts/rubicon/emily.html?pc=15178/93918&geo=na&co=us\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"NO\" width=\"0\" height=\"0\" style=\"height:0px;width:0px\"></iframe>\');})();<\/script>\n");",
"width": 728,
"height": 90
}]

As you can see adm field is js code. I'm not sure if there is any specification on acepted formats, but it seems to me that $$PREBID_GLOBAL$$.renderAd expects url or html code. As the result js code is inserted into the document and rendered as a text: https://ibb.co/epL735 (there are two banners on the page one under another).

I guess the easiest way is to wrap rubicon's response into "script" tags. But I'm not sure if rubicon always responds with js.

Improve error handling and reporting

Adapters return errors for two reasons.

  1. The user's request was bad (missing/invalid params)
  2. An actual error occurred (json.Marshal failed, the remote server was down, etc)

Right now, there's no way to distinguish between these errors, so we're logging everything.

We should introduce a specific error type for (1), and then only log errors which are not of that type. That should prevent log pollution with "error" messages like this:

W1003 19:47:37.726751      14 pbs_light.go:323] Error from bidder appnexus. Ignoring all bids: json: cannot unmarshal string into Go struct field appnexusParams.placementId of type int

User sync might have a negative impact on revenue

When user doesn't have advertisers cookies on prebid-server domain, instead of bids server responds with user sync url. And judging from google analytics data, this will happen a lot, for significant share of our pageviews there will be no bids, because users will have no advertisers cookies on prebid-server domain.

Of course when many publishers adopt prebid-server hosted on the same domain, this situation will became less common: user will get synced by one website using prebid server, and there won't be need to do that again on the other website. Though I'm not sure how much time it'll take prebid-server to gain sufficient popularity.

I might be missing something, mb the problem is not that significant as I imagine (I still haven't tested prebid server on production) or there is a better way to handle this. Still I would like to discuss the idea on how to alleviate this problem.

The way prebid server is integrated with prebid js, it seems to be quite simple to switch dynamically between client side and server side integration. So mb it would be a good idea to fallback to client side integration when the cookies are not synced. Possible solution:

  1. While initializing prebid, check user's cookies on publisher's domain for a flag that says that cookies were synchronized.
  2. If there is such flag, use prebid server, otherwise fallback to client side integration and do the user sync after the auction has ended. Then, set the cookies on publisher's domain so the next time the s2s integration would be used

Any feedback is appreciated. Coming from the publishers side I might be not have a sufficient understanding of how things work.

IndexExchange bidder isn't working?

When trying to get ads with indexExchange server bidder, I am getting no-bid, even after cookie sync. (tried 100 times)
when trying to get ads with the same placement ID and publisher ID with the JS bidder, I am always getting ads.

Server bid Params:
"bidder":"indexExchange",
"params":{
"id":"2",
"siteID":"179681"
}
Server bid Response:
{"tid":"9fa92356-47cb-4edd-930a-f7bb98b95d60","status":"OK","bidder_status":[{"bidder":"indexExchange","ad_unit":"doNotChange","response_time_ms":12,"no_bid":true,"debug":[{"request_uri":"http://appnexus-us-east.lb.indexww.com/transbidder?p=184932","request_body":"{\"id\":\"9fa92356-47cb-4edd-930a-f7bb98b95d60\",\"imp\":[{\"id\":\"doNotChange\",\"banner\":{\"format\":[{\"w\":300,\"h\":250}],\"w\":300,\"h\":250},\"tagid\":\"doNotChange\",\"secure\":1}],\"site\":{\"domain\":\"ultimate-guitar.com\",\"page\":\"https://www.ultimate-guitar.com/\",\"publisher\":{\"id\":\"179681\"}},\"device\":{\"ua\":\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36\",\"ip\":\"207.118.9.184\"},\"user\":{\"id\":\"2107298810938209085\",\"buyeruid\":\"WeyOv7lQJL4AADkpCfAAAABw%26240\"},\"at\":1,\"tmax\":1000,\"source\":{\"fd\":1,\"tid\":\"9fa92356-47cb-4edd-930a-f7bb98b95d60\"}}","status_code":204}]}]}

JS bid URL: https://as-sec.casalemedia.com/cygnus?v=7&fn=cygnus_index_parse_res&s=179681&r=%7B%22id%22%3A%22246919881%22%2C%22site%22%3A%7B%22page%22%3A%22https%3A%2F%2Fwww.ultimate-guitar.com%2F%22%7D%2C%22imp%22%3A%5B%7B%22id%22%3A%221%22%2C%20%22banner%22%3A%7B%22w%22%3A300%2C%22h%22%3A250%2C%22topframe%22%3A1%7D%2C%22ext%22%3A%20%7B%22sid%22%3A%222_1%22%2C%22siteID%22%3A179681%7D%7D%5D%7D&pid=pb0.25.0
JS bid Response:
cygnus_index_parse_res({"seatbid":[{"bid":[{"crid":"8668408","adomain":["vultr.com"],"adid":"8668408","impid":"1","cid":"1901418","id":"1","ext":{"dspid":105,"pricelevel":"_4","advbrandid":214517,"advbrand":"Vultr Holdings"},"adm":"<iframe src=\"https://a240.casalemedia.com/ifnotify?c=8444F8&r=EC63D02F&t=5A003E41&u=V2V5T3Y3bFFKTDRBQURrcENmQUFBQUJ3&m=07e6b1585fb1a67a0a078f1b157e9213&wp=6&aid=8EA8C040786A0C9D&tid=F582&s=2BDE1&cp=0.06&n=www.ultimate-guitar.com&pr=xx&sid=379c&epr=246919881\" width=\"0\" height=\"0\" frameborder=\"0\" scrolling=\"no\" style=\"display:none;\" marginheight=\"0\" marginwidth=\"0\"></iframe>\n <div style=\"text-align: center;\">\n <script type=\"text/javascript\">\n adroll_width = 300;\n adroll_height = 250;\n adroll_a_id = \"SV2SF2P76NFIPN7N5SDPHX\";\n adroll_s_id = \"ISRFVJ4CYZEJLCRYPCOWW4\";\n adroll_c_id = \"MKPW6JTMURCVDDC6WM6KR5\";\n adroll_render_link = false;\n adroll_ext_network = \"index\";\n adroll_subnetwork = \"r\";\n adroll_ad_payload = \"__HIAg8BkwHaAf3IAfkAAY2QTWjTYACG08-uniaIN0_Kdm1JvqT5KVrp1qU_mX8rrW0RY36-dJ-kqbbJot7E024KHnReRIabP0xFqJtHB3bYfFHw5BAF2UH0oAfxIChGELx48Pw-Lzw8b6nxeL_YlI8cnCiWjqblRj0_Xecma8ca7O5-vqoUc0qtUSrnKsV8daLA1YSZKbE_lZeFaqMgl8uF8qHDJaVSmZRlIUM-nty3bHRsG5uomx1mX848WbjWVA6sQZoRGBpyqj-LXWTjnqtC1UU9Vx7sBFvvPz28NP_zy-rag5V3z16tjChPj4fU3tR2bwF5ST-ikox29l7XcxzstFSeEZlg_rHv-ynPdnFbc1Gy5WFX66aMTju4HjapbC648ehf481EsJgLni8hmjc0XguGg0T8ckjt39gRBL9NXt_6_GHx_rf-6h-T4PtgdNuFZTSHzY6juiRzNfhxhU7xkMTGNqkxBDXJFNK8pFumYEmMxolsGrIShKaeNkQug20SG_9PENRDKrF1m4DGJrWnd-a011LnBFVQGahG6jp2kPk3HwHNdQJORMEJaIfUxd4pAs6HsdGvWQI2oj-ts5YmWTwn8gbDQV40GNqUBJPVeZplGZqA4ZJ5ztHa2CAgfEPtwkkaipYuIQ2JhiggyOskPr1ORu7c7aIoXwu5UXySmA0p8ML4BaxjDEqW5v-pYCi9Lvk\";\n adroll_url_macro = \"ultimate-guitar.com\";\n adroll_cpm_macro = \"WgA-QQAAAADh4IS8qL3BxquQiFxff6jLNMv_HA\";\n adroll_c_macro = \"\";\n adroll_imp_macros = undefined;\n adroll_win_notif = \"index-winners-eu-west-1-rtb.adroll.com/win/index?c=EDF7UYGFJJGJNOIKSSCFF7;bid=27853944882346095040826299410739556;payload=__HIAuQAARXSO67jVACA4auRWAA7YAPnym-fNEh24mccx_EjfjTW8Tt2_LZjJxsY0bGFsIDRTINgA0xHQzdCQkJ0NKwAcWn_6iv-90_05eUbLCIztMkYCjIxThEMjHEs2bAJGTEYSeLYM_r89bu__vz70_ff_fvPTz9__PDHL799-OoZ_yjsRNbxJVFVJVU_KnvL2ooi-yx-f8HTmAAkBQiaBDjcAJx9TWewpOME8Ne4rbt5Sl9RjR5tg5bx__Lsvrz7HIaywuWQrxYZcRmpcFnFcT5xst3S9rmthEEwNxek6Fi4T2pcvfU2vT8PHd47Ye3BYXAx0ycZkKig6qNIbSdWwAhqLzdmaiooMrBV7NVJ5snIz0R5oXlOJBwZa7lsVbaFJGgrBY0FaI8FRkyzplroXxUYN9uLW_bR1Syx0EvjGsEYsUca3PrWibF-cqyh2Y9nmCCNw30r2EdCeA1SEfWH_ApnEfbyTkVnqw7Mq-IcQDPdyUg2R4me23rF3-gUdKPSPXPMeApaU-K7e18CzXPtsumnLe44m_PKjrx4uxkPAupG-LiDAeX0mXzgzFRuC6OkMoecTCJaRNqywaVqV2JzZb14fYyTaKVCcM3DQb6Be2MkjRjFuHSnXdFjGHSwy5mSS9RF27cJ0iKjIC6Maq41F1jUZYx5-SzlmA3jqpzITW7y2kb3ppAJJXkNhwwwBbApY8oTfRYD25IfPlPOF_OhZ8VaOOxaHWxJ76uV33mqqO9hJ861L9SHtb2O0qBYxR6lenWacwKzpHBjCLq52w8iXlM7SR3cJgEIw3Gt4cXzSUKFxJQdURkVKS-ynpzUo3kQVKRkzppQHTcy8phIN_ekEG0JppH2a9deGphHu9OZwt4gDq8a5zTorlLQc1Sg0o9JRsSWyzPBmJaqQpfOn5UToRaXPL8fx8oY8KM5szGLlNZbZtrx7lbNYR08aBSP1nIn9C59A52_vWy0W_WcntkP0_hr8PIt9x8;price_cpm_dollars=\";\n </script>\n <script type=\"text/javascript\">\n var _url_info =\n ('https:' == 'https:'\n ? {adserver_hostname:\"s.adroll.com\", protocol:\"https://\"}\n : {adserver_hostname:\"a.adroll.com\", protocol:\"http://\"}\n );\n\n (function () {\n document.write('<scr'+'ipt type=\"text/javascript\" src=\"' + _url_info.protocol + _url_info.adserver_hostname + '/j/rolling.js\"></scr'+'ipt>');\n }());\n </script>\n <script type=\"text/javascript\" src=\"https://q.adrta.com/aa.js?cb=f0Cw7uExxCsEbjIMss591SwujsCvHenih6bru7QibjE=#ar;paid=ar;avid=DUKHAKVYIJASHDUBG4V7RE;caid=EDF7UYGFJJGJNOIKSSCFF7;plid=HZFPMBHIQ5FYXDLX4CVWY3;siteId=ultimate-guitar.com;kv1=300x250;publisherId=184163;kv2=https%3A%2F%2Fwww.ultimate-guitar.com;kv3=0b3fa9f6486c14268c10d97d3b603310;kv4=31.154.177.226;kv7=CmgnFR4;kv11=27853944882346095040826299410739556\"></script><noscript><img src=\"https://adrta.com/i?clid=ar&paid=ar&avid=DUKHAKVYIJASHDUBG4V7RE&caid=EDF7UYGFJJGJNOIKSSCFF7&plid=HZFPMBHIQ5FYXDLX4CVWY3&siteId=ultimate-guitar.com&kv1=300x250&publisherId=184163&kv2=https%3A%2F%2Fwww.ultimate-guitar.com&kv3=0b3fa9f6486c14268c10d97d3b603310&kv4=31.154.177.226&kv7=CmgnFR4&kv11=27853944882346095040826299410739556\"></noscript>\n </div>\n "}],"seat":"1"}],"cur":"USD","id":"246919881"});

OpenRTB support: Bid cache

The /openrtb2/auction endpoint needs to support sending bids to the cache.

Right now, I don't think we have a good enough sense of the cache's use-cases to build a quality feature here. This is the fundamental tradeoff:

  • Bids cached unnecessarily waste computing resources of the PBS host.
  • Bids not cached limits the publisher, to varying extends depending on the context.

If the cache behavior needs to be "chosen", because different behaviors have equal merit, then we have two options:

  • If the choice lies with the publisher, then we can add to the OpenRTB request.ext
  • If the choice lies with the host, we can add to the PBS Config

However... I also want to note the following, which we should take very seriously:

  • Each "choice" adds code complexity, which increases the time of all future development.
  • Choices are easy to add, but very hard to remove. Users will grow to depend on them.

To grow this project smoothly, our goal should be to identify the fewest "choices" necessary to encourage adoption on both sides.


These are things to consider:

  • The Prebid.JS philosophy is: "the publisher ultimately decides which bid wins." If PBS caches the bids, Prebid auctions will have overall better performance. However, it is possible for JS expose an API which caches only the bid which the publisher chooses.

  • The prebid-mobile philosophy is: "App code updates are expensive. It's better if the Server does everything, including picking the winning bid." As far as I can tell, there's no room for negotiation here.

  • @DucChau mentioned that Rubicon wanted an option to cache just the vast XML. Currently PBS caches a JSON blob with other info too.

  • There's some ongoing debate about the cache's usefulness for certain types of bids in #199.


These are some deficiencies in the /auction endpoint's cache behavior. Ideally, we can do it better in openrtb2/auction.

  • Bids aren't cached until the auction has completed. So if any bidder in the auction times out, then the cache calls do too. This basically wastes the auction on mobile, since it requires the bid to be cached.
  • @DucChau noted that /auction doesn't let you cache just the vast XML. It requires the JSON blob. If this is a valid use-case, then the /openrtb2/auction API should to support this too.

@bretg and @mjacobsonny might want to discuss this. At the moment, I don't think it's fleshed out well enough for much useful engineering work to be done.

Adding Support for Appnexus Aliases

Not sure if you guys want this on the server-side or client-side, but there's currently no support for the various appnexus aliases adapters in here.

Fix unreliable tests

Travis builds are failing intermittently. Some sample output of the failures is below.

ok  	github.com/prebid/prebid-server	0.078s
body {"account_id":"","tid":"","cache_markup":0,"sort_bids":0,"max_key_length":0,"secure":0,"timeout_millis":0,"ad_units":[{"code":"first-tag","is_top_frame":0,"sizes":[{"w":300,"h":250}],"bids":[{"bidder":"audienceNetwork","bid_id":"random-id-from-pbjs-0","params":{"placementId":"987654321_999998888"}}],"config_id":""},{"code":"second-tag","is_top_frame":0,"sizes":[{"w":300,"h":250}],"bids":[{"bidder":"audienceNetwork","bid_id":"random-id-from-pbjs-1","params":{"placementId":"987654321_66775544"}}],"config_id":""}],"is_debug":false,"app":null,"device":null,"user":null,"Start":"0001-01-01T00:00:00Z"}
--- FAIL: TestFacebookBasicResponse (0.01s)
	facebook_test.go:273: Should not have gotten an error: context deadline exceeded
&{ [{div-adunit-1 0xc4203c2800 <nil> <nil>  0 1001 0  0 [] <nil> []}] 0xc420331860 <nil> 0xc420080d00 0xc4202a6280 0 1 0 [] 0 [] [] [] 0xc4201dd180 <nil> []}
body {"account_id":"","tid":"","cache_markup":0,"sort_bids":0,"max_key_length":0,"secure":0,"timeout_millis":0,"ad_units":[{"code":"first-tag","is_top_frame":0,"sizes":[{"w":300,"h":600},{"w":300,"h":250}],"bids":[{"bidder":"rubicon","bid_id":"random-id-from-pbjs-0","params":{"zoneId":8394,"siteId":283282,"accountId":7891}}],"config_id":""},{"code":"second-tag","is_top_frame":0,"sizes":[{"w":300,"h":600},{"w":300,"h":250}],"bids":[{"bidder":"rubicon","bid_id":"random-id-from-pbjs-1","params":{"zoneId":8395,"siteId":283282,"accountId":7891}}],"config_id":""}],"is_debug":false,"app":null,"device":null,"user":null,"Start":"0001-01-01T00:00:00Z"}
Request {"id":"","imp":[{"id":"second-tag","banner":{"w":300,"h":600,"format":[{"w":300,"h":600},{"w":300,"h":250}],"ext":{"rp":{"size_id":10,"alt_size_ids":[15],"mime":"text/html"}}},"ext":{"rp":{"zone_id":8395}}}],"site":{"domain":"nytimes.com","page":"https://www.nytimes.com/2017/05/04/movies/guardians-of-the-galaxy-2-review-chris-pratt.html?hpw\u0026rref=movies\u0026action=click\u0026pgtype=Homepage\u0026module=well-region\u0026region=bottom-well\u0026WT.nav=bottom-well\u0026_r=0","publisher":{"ext":{"rp":{"account_id":7891}}},"ext":{"rp":{"site_id":283282}}},"device":{"ua":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/603.1.30 (KHTML, like Gecko) Version/10.1 Safari/603.1.30","ip":"25.91.96.36"},"user":{"buyeruid":"need-an-actual-fb-id"},"at":1,"source":{"fd":1}}
Request {"id":"","imp":[{"id":"first-tag","banner":{"w":300,"h":600,"format":[{"w":300,"h":600},{"w":300,"h":250}],"ext":{"rp":{"size_id":10,"alt_size_ids":[15],"mime":"text/html"}}},"ext":{"rp":{"zone_id":8394}}}],"site":{"domain":"nytimes.com","page":"https://www.nytimes.com/2017/05/04/movies/guardians-of-the-galaxy-2-review-chris-pratt.html?hpw\u0026rref=movies\u0026action=click\u0026pgtype=Homepage\u0026module=well-region\u0026region=bottom-well\u0026WT.nav=bottom-well\u0026_r=0","publisher":{"ext":{"rp":{"account_id":7891}}},"ext":{"rp":{"site_id":283282}}},"device":{"ua":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/603.1.30 (KHTML, like Gecko) Version/10.1 Safari/603.1.30","ip":"25.91.96.36"},"user":{"buyeruid":"need-an-actual-fb-id"},"at":1,"source":{"fd":1}}
Request {"id":"","imp":[{"id":"first-tag","banner":{"w":300,"h":600,"format":[{"w":300,"h":600},{"w":300,"h":250}],"ext":{"rp":{"size_id":10,"alt_size_ids":[15],"mime":"text/html"}}},"ext":{"rp":{"zone_id":8394}}}],"site":{"domain":"nytimes.com","page":"https://www.nytimes.com/2017/05/04/movies/guardians-of-the-galaxy-2-review-chris-pratt.html?hpw\u0026rref=movies\u0026action=click\u0026pgtype=Homepage\u0026module=well-region\u0026region=bottom-well\u0026WT.nav=bottom-well\u0026_r=0","publisher":{"ext":{"rp":{"account_id":7891}}},"ext":{"rp":{"site_id":283282}}},"device":{"ua":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/603.1.30 (KHTML, like Gecko) Version/10.1 Safari/603.1.30","ip":"25.91.96.36"},"user":{"buyeruid":"need-an-actual-fb-id"},"at":1,"source":{"fd":1}}
Request {"id":"","imp":[{"id":"second-tag","banner":{"w":300,"h":600,"format":[{"w":300,"h":600},{"w":300,"h":250}],"ext":{"rp":{"size_id":10,"alt_size_ids":[15],"mime":"text/html"}}},"ext":{"rp":{"zone_id":8395}}}],"site":{"domain":"nytimes.com","page":"https://www.nytimes.com/2017/05/04/movies/guardians-of-the-galaxy-2-review-chris-pratt.html?hpw\u0026rref=movies\u0026action=click\u0026pgtype=Homepage\u0026module=well-region\u0026region=bottom-well\u0026WT.nav=bottom-well\u0026_r=0","publisher":{"ext":{"rp":{"account_id":7891}}},"ext":{"rp":{"site_id":283282}}},"device":{"ua":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/603.1.30 (KHTML, like Gecko) Version/10.1 Safari/603.1.30","ip":"25.91.96.36"},"user":{"buyeruid":"need-an-actual-fb-id"},"at":1,"source":{"fd":1}}
Error in rubicon adapter: context deadline exceededError in rubicon adapter: context deadline exceededFAIL

Expected: Unit tests always pass or fail reliably.

When request is proxied to prebid-ssl.northeurope.cloudapp.azure.com wrong X-Real-Ip is set

When I query prebid-server, request goes to prebid.adnxs.com and then it gets redirected to prebid-ssl.northeurope.cloudapp.azure.com . For some reason prebid proxy adds wrong X-Real-Ip header (11.244.1.1). As the result prebid server erroneously uses what I suppose to be datacenter's ip as my own. Here is the response I get from prebid's debug page https://prebid.adnxs.com/pbs/v1/ip

User Agent: Type: Browser
Name: Chrome
Version: 58.0.3029
OS: GNU/Linux
OSVersion: 0.0.0
Security: Unknown security
Mobile: false
Tablet: false

IP: 11.244.5.1

Port: 32994

Forwarded for: 11.244.1.1

X-Forwarded-Host: [prebid.adnxs.com]

Accept: [text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8]

Accept-Language: [en-US,en;q=0.8,ru;q=0.6]

X-Real-Ip: [11.244.1.1]

X-Forwarded-For: [11.244.1.1]

X-Forwarded-Proto: [https]

Cache-Control: [max-age=0]

Upgrade-Insecure-Requests: [1]

User-Agent: [Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36]

Accept-Encoding: [gzip, deflate, sdch, br]

The wrong ip is also passed in device section of RTB request and as the result I see ads targeted to US only (while my ip is European).

This happens only for european IPs and doesn't happen when I query prebid.adnxs.com from US-based ip. In the latter case, my ip is passed correctly in X-Real-Ip header.

Benchmark json-iterator as a possible replacement for the standard json parser

The library https://github.com/json-iterator/go claims to be much faster than the standard json parser shipped with go, yet will work as a drop in replacement. We do a lot of JOSN parsing in prebid-server. (Really most of the code is just parsing JSON, manipulating the data structures, and then publishing the modified JSON) So if there really is a big performance gain with this library, we should see a definite improvement in this project.

Mitigate race conditions in Adapters

In this project's short lifetime, we've already started to see how easy it is to introduce race conditions (#122, #140).

This is a problem, because:

  1. The project has no automated way to catch them. Only code review & comments.
  2. Adapters are run in their own goroutine, so segfaults crash the entire process.

Programmers make mistakes, and anything which isn't caught automatically will happen in the future. As this project becomes more active, this will become a bigger issue.

There are a few options for how we can mitigate these issues.

  1. Copy every object we send into the Adapters' goroutines, so that data races aren't possible.
  2. Add "good adapter citizen" tests which run on all adapters, and make sure they don't mutate shared state.
  3. Change the Types we send into Adapter.Call() so that adapters can't mutate things they're not allowed to. For example, make the PBSRequest state private and expose methods which only return immutable data.

Anyone have preferences, or other ideas?

OpenRTB support: Facebook adapter

Now that #196 has landed, the /openrtb2/auction endpoint is (technically) live... with one bidder. This deprecates /auction, and moves the project towards #150.

The next step is to update existing adapters to support it. Documentation can be found here.

This issue is for making or updating the Facebook adapter to support OpenRTB.

appnexus adapter segfault

Seen while running in production on version 0.1.0 with the following stack trace:

panic: runtime error: index out of range

goroutine 755733804 [running]:
github.com/prebid/prebid-server/adapters.(*AppNexusAdapter).Call(0xc460655300, 0xc74de0, 0xc479016d20, 0xc498c4c700, 0xc4897e9c00, 0x0, 0x0, 0x0, 0x0, 0x0)
        /Users/dbemiller/go/src/github.com/prebid/prebid-server/adapters/appnexus.go:90 +0xc60
main.auction.func1(0xc75fa0, 0xc460655300, 0xc74de0, 0xc479016d20, 0xc498c4c700, 0xc46052bf00, 0xc42012ac00, 0xc420207040, 0xc479016f60, 0xc4897e9c00)
        /Users/dbemiller/go/src/github.com/prebid/prebid-server/pbs_light.go:312 +0xa0
created by main.auction
        /Users/dbemiller/go/src/github.com/prebid/prebid-server/pbs_light.go:351 +0x6bb

This seems to happen very rarely... but should still be fixed.

Document current status of demand adapters

All of the demand adapters support different media-types (banner, video, native) as well as supply types (web, mobile). This should definitely be tracked somewhere so that it is easy for developers to know without digging through the code what an adapter supports.

@dbemiller is thinking to do this through APIs.

Might need to add a GET /bidders endpoint which returns something simpler, like:

[
"appnexus",
"rubicon",
...
]
And maybe a /bidders/media-types which returns something like:

{
"appnexus": ["banner", "video"],
"index": ["banner"]
}
Basic motivation is: docs fall out of date all the time. Since we're running a server anyway, we should be able to generate these things from the code, so they're guaranteed to be up to date.

It'll also help with the fact that different companies might deploy different versions of PBS... so the features might be different depending on whose deploy you're using.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.